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ABSTRACT 


The Department of Defense relies on modeling and simulation for a variety of 
purposes, including joint exercise training, developing and evaluating doctrine and 
tactics, and studying weapon system effectiveness. Advances in technology have made 
the achievement of technically and visually accurate simulations possible, but little has 
been done to present realistic scenarios while supporting user interaction. This 
dissertation describes a multi-agent interactive simulation engine for generating 
interactive scenarios or stories. A general-purpose multi-agent system simulation 
architecture, called a Connector-based Multi-Agent System (CMAS) is developed and 
presented, along with a software agent communication and coordination mechanism. In 
this architecture, stories are generated through discovery as a by-product of agent 
interactions, rather than being fixed in advance. The ensuing story adapts to the user’s 
interventions and is closely aligned to the goals of the agents. The multi-agent system 
design of the story engine has resulted in a data-driven simulation engine, which is 
domain independent and highly scalable. 

The story engine is fielded as the underlying simulation engine behind the U.S. 
Army’s America’s Army: Soldiers project. The instantiation of the story engine as it 
applies to Soldiers is presented. As a component of Soldiers, the story engine is an 
integral module in an interactive story generation system. 


V 



THIS PAGE INTENTIONALLY LEET BLANK 


VI 



TABLE OF CONTENTS 


L INTRODUCTION.1 

A, THESIS STATEMENT.1 

B, MOTIVATION.2 

C, PROBLEM STATEMENT.2 

1. Definition of Interactive Story.3 

2. Interactive Story.4 

3. Summary.7 

D, APPROACH.7 

E, CONTRIBUTIONS.9 

F, DISSERTATION ORGANIZATION.10 

II. RELATED WORK.13 

A, INTRODUCTION.13 

B, NARRATIVE INTELLIGENCE.14 

1, Artificial Intelligence Story Systems.14 

2, Story-Understanding Systems.15 

3, Story-Telling Systems.16 

C, NARRATIVE THEORY AND SCREENPLAY STRUCTURES.16 

D, INTERACTIVE ENTERTAINMENT.19 

E, INTERACTIVE STORIES.22 

1, Centralized Drama Manager.24 

a. Believable Agents.25 

b. Drama Manager.27 

2, Dramatic Beats.30 

3, Directed Improvisation.31 

4, Verb-Centric Interactive Story-Telling.32 

5, Interactive Stories with Strong Pedagogical Goals.33 

a. A Story Deconstruction Approach to Interactive Drama „34 

b. Predefined Networks with Confined Freeplay.35 

F, AGENT-BASED PLANNING SYSTEMS.37 

G, SUMMARY.39 

III. MULTI-AGENT SYSTEMS.41 

A. INTRODUCTION.41 

B. MULTI-AGENT SYSTEM DEFINITION.42 

1. Environment.42 

2. Objects.43 

3. Agents.43 

4. Relations.44 

5. Operations.44 

6. Laws.45 

C. SUMMARY.45 

IV. MULTI-AGENT SYSTEM RESEARCH.47 

A. INTRODUCTION.47 

B. SEMI-FLUID SOFTWARE STRUCTURE.47 

vii 













































1, Indirect Solutions - A Design Paradigm Shift.48 

C. COMPOSITE AGENTS.49 

D. REACTIVE AGENTS AND GOAL MANAGEMENT.50 

E. CONNECTORS.52 

1, The Biological Inspiration Behind Connectors.53 

2, Connector Definition.57 

a. Graphical Notation.58 

h. Host Entity.59 

c. Control Function.60 

d. State.62 

e. Mode.63 

f. Type.64 

g. Value.64 

h. Action set.65 

F. TICKETS.66 

1. Ticket Control Function.67 

2. Ticket Frames.67 

3. Ticket Connectors.69 

G. SUMMARY.70 

V. CONNECTOR-BASED MULTI-AGENT SYSTEM ARCHITECTURE.73 

A. INTRODUCTION.73 

B. CONNECTOR-BASED MULTI-AGENT SYSTEM.73 

C. ENVIRONMENT.74 

D. OBJECTS.75 

E. CONNECTOR-BASED COMPOSITE AGENTS.77 

1. Symbolic Constructor Agents and Einner.78 

2. Tickets and Actions.79 

a. References.80 

h. Actions.81 

c. Signal Cascades.82 

3. Agent Connector Sets.83 

4. Reactive Agents and Goals.84 

F. OPERATIONS.86 

1. Connecting.87 

G. RELATIONS.89 

H. LAWS.91 

I. CONNECTOR SET.93 

J. MANAGING THE COMPLEXITY OF AGENT INTERACTIONS.93 

K. CONNECTOR-BASED SIMULATION MODEL.94 

L. SUMMARY.95 

VL INTERACTIVE STORY GENERATION SYSTEM.97 

A. INTRODUCTION.97 

B. STORY ONTOLOGY.98 

1. Story World, Story and Story Line.98 

2, Story World as a CMAS.100 

C. STORY ENGINE CMAS.102 

viii 

















































1, Character Agents.103 

a. Goals and Actions.104 

b. Most Favorable C onnection.105 

2, Scene Agents.106 

a. Scenes.107 

b. Inner Environment.108 

c. Tickets.108 

d. Connectors: External and Internal.109 

e. Interactions.109 

3, Character Agent to Scene Agent Connections.114 

D. LOGICALLY CONNECTED, GOAL-DIRECTED AND SOUND 

STORYLINES.117 

1. Logically Connected and Goal-Directed Story Lines.117 

2, Sound Story Lines.118 

E. RUN-TIME ANALYSIS AND SCALABILITY OF STORY LINE 

GENERATION.119 

1. Run-time Analysis.120 

2, Story World Dimension.120 

F. SCENE RENDERING SUBSYSTEM.121 

1. Animation Engine.122 

2. Location Generator.122 

3. Text-to-Voice.124 

G. GRAPHICAL INTERFACE.125 

H. SUMMARY.126 

VII. CMAS DATA SET IMPLEMENTATION - AMERICA’S ARMY: 

SOLDIERS.127 

A, INTRODUCTION.127 

B, AMERICA’S ARMY: SOLDIERS.127 

C, INTERACTIVE STORY GENERATION SYSTEM DATA.128 

D, CHARACTER DEFINITIONS.129 

1. Actors, Characters and Roles.130 

2. Core Values and Resources.131 

3, Goals.133 

4, Tickets: Army Training Progression.135 

E, CONNECTOR SET.136 

F, SCENE DEFINITIONS.138 

1. Exemplar Scene: Physical Training.138 

2, Exemplar Story.140 

G, OBJECTS.140 

H, SUMMARY.141 

VIIL CONCLUSIONS AND RECOMMENDATIONS.143 

A. CONCLUSIONS.143 

B. APPLICATIONS.144 

1. Educational Gaming.144 

2, Scenario-Based Training.144 

C. FUTURE WORK.145 














































1, Narrative Structure.145 

2, Potential Outcome Modeling.145 

3, Improved Cognitive Architecture.145 

4, Relations.146 

5, Generalized Connecting.146 

6, Agent Learning.146 

D, SUMMARY.147 

APPENDIX A. EXAMPLE STORY.149 

APPENDIX B, ACRONYMS AND ABBREVIATIONS.157 

LIST OF REFERENCES.159 

INITIAL DISTRIBUTION LIST.165 


X 














LIST OF FIGURES 


Figure 1. Story World Diagram.6 

Figure 2. Interaetive Story Generation System.9 

Figure 3. Dramatie Strueture.17 

Figure 4. David Siegel’s Nine-Aet Strueture from [Siegel, 2001].17 

Figure 5. Vogler’s Hero’s Journey Model from [Voytilla, 1999].19 

Figure 6. Oz Projeet Arehiteeture from [Mateas, 1997].25 

Figure 7. StoryNet Arehiteeture from [Swartout et al, 2001].37 

Figure 8. PLANGENT Planning Steps from [Oshuga et al., 1997].39 

Figure 9. Composite Agent.50 

Figure 10. Reaetive Agent.52 

Figure 11. Cell Signaling Methods from [Cooper, 1997].54 

Figure 12. Cells Responding to Combinations of Extraeellular Signals from [Alberts et 

al, 2002].55 

Eigure 13. Intraeellular Signal Caseading from [Alberts et al., 2002].56 

Eigure 14. Graphieal Depletion of Conneetors.59 

Eigure 15. Conneetor Attaehed to a Host Agent.60 

Eigure 16. Predator/Prey Control Eunetion.61 

Eigure 17. Conneetor State Transition Diagram.62 

Eigure 18. Charaeter Agent Energy Conneetor.65 

Eigure 19. Example Conneetor-based Tieket.69 

Eigure 20. Tieket and Aetion State Dynamieally Maintained with Conneetors.70 

Eigure 21. Conneetor-based MAS.74 

Eigure 22. CMAS Objeet.76 

Eigure 23. Conneetor-based Composite Agent.78 

Eigure 24. Eat Aetion Binding to Eood Objeet.82 

Eigure 25. Signal Caseade.83 

Eigure 26. Reaetive Agent Goal Strueture.86 

Eigure 27. Resouree Sharing Relationship.91 

Eigure 28. CMAS Simulation Model.95 

Eigure 29. Interaetive Story Generation System.97 

Eigure 30. Three Pigs and a Wolf Story World.99 

Eigure 31. Interaetive Story Generation System with Agents and Media.103 

Eigure 32. Charaeter Agent Goals with Bound Aetions.106 

Eigure 33. Seene Agent.107 

Eigure 34. Greet / Response Interaetion Sequenee.113 

Eigure 35. Charaeter Agent to Seene Agent Conneetion.115 

Eigure 36. Story Eine Generated from a Story Engine CMAS Story World.116 

Eigure 37. Eoeation Templates.124 

Eigure 38. Text-to-Voiee Generative Grammar.125 

Eigure 39. AA: Soldiers Interaetive Story Generation System Data Set.129 

Eigure 40. AA; Soldiers Charaeter Definition Sereen.131 

Eigure 41. Charaeter Personality Defined by Army Core Values.132 

Eigure 42. Charaeter Aptitude Expressed in Terms of Resourees.132 

Eigure 43. Goal Emphasis and Progress.134 


XI 














































Figure 44. Goal Description Including Prerequisites, Requirements and Results.135 

Figure 45. Goal Hierarchy Formed from Goal Prerequisites.135 

Figure 46. U.S. Army Career Training Progression.136 

Figure 47. Basic Physical Training: Run Qualification Scene.139 


xii 







LIST OF TABLES 


Table 1. Story Engine CMAS Aetions.110 

Table 2. Seene Rendering Actions.111 

Table 3. Character’s List of Potential Goals.133 

Table 4. Army Training Progression.136 

Table 5. Connectors Describing Character Personality, Aptitude and Goals.137 

Table 6. Connectors Describing Army Career Progression and Locations.138 

Table 7. Interactions for Basic Physical Training: Run Qualification Scene.140 

Table 8. AA: Soldiers Story World Objects.141 


xiii 











THIS PAGE INTENTIONALLY LEET BLANK 


XIV 



LIST OF EQUATIONS 


Equation 1. Multi-Agent System from [Ferber, 1999].42 

Equation 2. Action Set for Agent Type a at Time t.45 

Equation 3. Goal Definition.51 

Equation 4. Connector Definition.58 

Equation 5. Connector Current Value Function (O).64 

Equation 6. Requirement for Two Connectors to Connect - conn(ci, C 2 ).66 

Equation 7. Ticket Definition.67 

Equation 8. Connector-based Multi-Agent System (CMAS) Definition.74 

Equation 9. CMAS Environment.75 

Equation 10. CMAS Object Definition.76 

Equation 11. CMAS Object Set (O).76 

Equation 12. CMAS Objects at Time t (Ot).76 

Equation 13. Connector-based Composite Agent Definition.77 

Equation 14. Agent Inner Environment (Einner).79 

Equation 15. Reference Definition.80 

Equation 16. Reference Action to Ticket Connection.80 

Equation 17. Action Definition.81 

Equation 18. Reference Action to Action Connection.81 

Equation 19. Reactive Agent Definition.84 

Equation 20. Agent Goal Set (G).85 

Equation 21. CMAS Operations Set (Q).87 

Equation 22. Agent-to-Agent Connection Conditions - conn(ai, aj).87 

Equation 23. Agent-to-Object Connection Conditions - conn(ai, oj).88 

Equation 24. Awareness Set (W).89 

Equation 25. Candidate Connection Set (W').89 

Equation 26. Most Favorable Connection (mfc).89 

Equation 27. CMAS Connector Set (C).93 

Equation 28. Story World Definition.98 

Equation 29. CMAS Generating Function (S).100 

Equation 30. Character Agent Most Favorable Connection Function.105 

Equation 31. Interaction Definition.112 

Equation 32. Interaction-to-Interaction Connection - conn(intXj, intxi).112 


XV 



































THIS PAGE INTENTIONALLY LEET BLANK 


XVI 



ACKNOWLEDGMENTS 


As always in an endeavor such as this, a number of people played a role in seeing 
the work from inception to conclusion. First I must recognize John Hiles—this 
dissertation would have been impossible without his guidance and inspiration, for which I 
am deeply grateful. His insights were crucial. I owe special gratitude as well to Mike 
Zyda. A true visionary, his recommendation led me down this path of interactive 
narrative. Ted Lewis deserves my thanks for his patience as I put thoughts on paper. 
Always a constructive listener, this document benefited greatly from his discerning 
feedback, and I benefited from our many hours of discussion. To Mike Capps, Rudy 
Darken, and Don Brutzman, my appreciation for the direction provided and experience 
shared. 

This work progressed under the ready support of faculty, staff, and students of the 
Computer Science Department and MOVES Institute, with a special thanks to Margaret 
Davis for her assistance with the graphics that appear here. I wish to recognize my 
fellow doctoral students, who served as a well of ideas and support: Curt Blais, Simon 
Goerger, Perry McDowell, Mike VanPutte, and Joerg Wellbrink. 

The US Army’s Office of Economic and Manpower Assessment (OEMA) 
provided the inspiration behind the America’s Army project. I am indebted to Eieutenant 
Colonel Casey Wardynski and the rest of the OEMA team for their help and belief in the 
project. The America’s Army: Soldiers development team of Euke Ahearn, Mike Bailey, 
Neal Elzenga, Randy Jones, James Marsh, and John Mason is an inspired and dedicated 
team with a vision of producing a radically new genre of game; it was a privilege 
working with them. 

To my family—Carrie, Chris, Melissa, and Alan—it is hopeless to try to express 
the depth of my gratitude for your love and encouragement. This dissertation is dedicated 
to you. 


xvii 



THIS PAGE INTENTIONALLY LEET BLANK 



I. INTRODUCTION 



“I like a good story well told. That is the reason I am sometimes forced to 
tell them myself.” 

Mark Twain, 1907 


A. THESIS STATEMENT 

Highly dynamic, interactive stories can be generated by a multi-agent system as a 
by-product of agent interactions. These stories are sensitive to user input, the actions, 
goals and internal states of autonomous characters, and the global state of the 
environment. By formulating the multi-agent system as a data-driven architecture, a 
domain independent story system can be developed that generates logically connected, 
goal-directed stories that are sound with respect to the domain of interest. 
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B, MOTIVATION 

In 1997 the National Research Council (NRC) issued a report that specified a 
joint research agenda for defense and entertainment modeling and simulation [NRC, 
1997]. The NRC report provides a guide as to what research and development needs to 
be done to develop future interactive entertainment and defense modeling and simulation 
systems. For the entertainment industry, as stated in the NRC report, modeling and 
simulation technology lies at the heart of video games, theme park attractions, and 
entertainment centers. The Department of Defense (DoD) uses modeling and simulation 
for a variety of purposes, such as to conduct joint training exercises, develop and evaluate 
new doctrine and tactics, analyze alternative force structures, and study the effectiveness 
of new weapons systems. Advances in information technology have lowered the cost of 
computer-based models and simulation, making modeling and simulation a cost-effective 
alternative to live training and exercises. 

The NRC report identified five areas as having potential for greater collaboration 
between the entertainment industry and defense, one of which included computer¬ 
generated characters. It is in this area, specifically the control of such characters within 
the bounds of an interactive story, that this research focuses. 

C. PROBLEM STATEMENT 

An interactive story system should support a complicated story world comprised 
of characters and props, along with locations where events occur and characters interact 
with each other, the props, and their surroundings. The interactions should not be 
random, nor should the characters be free to act as they wish without regard to the rest of 
the story world. There are constraints on what can be done and how interactions occur. 
At the same time, it must be adaptable to multiple domains, whether it be presenting 
training scenarios in a ground combat simulation or an action-adventure story. To make 
this story system interactive, allowances must be made for a user to intervene as the story 
progresses, while at the same time ensuring the story remains logically connected and 
aligned with the protagonist’s goals. 

Current approaches based on artificial intelligence top-down planning techniques 

can support complicated plots with a diverse set of story characters, but they are 

extremely domain-knowledge specific. All possible character-to-character and character- 
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to-environment interactions must be defined in advance along with a predefined story 
script or plan. Extensive time and effort is required to generate knowledge bases and 
dependency networks for each new story [Schank and Abelson, 1977], [Young, 1999]. 
Algorithmic approaches using tree or graph structures to organize story events provide a 
domain independent methodology, but for complicated stories, these approaches are 
overcome by the combinatorial problem of evaluating all possible plots each time an 
event occurs [Weyhrauch, 1997]. Similarly, if the story is highly scripted and the 
corresponding graph structure simplified, the resulting stories are akin to highly 
immersive hypertext documents where the branch points are tightly constrained in order 
to meet pedagogical goals or conform closely to the author’s story line [Marsella et al., 
2000], [Swartout et al., 2001]. 

Thus, previous approaches have taken a largely top-down view of creating a story, 
where the author must in some way anticipate all actions the interactive user might take 
at each step of the story. The problem of creating a general interactive story system is 
one of developing an architecture that scales well to large story worlds where all possible 
interactions may not be known a priori, while at the same time remaining domain 
independent. 

1. Definition of Interactive Story 

The following definition of interactive story is taken from Brenda Laurel’s 

dissertation Toward the Design of a Computer-Based Interactive Fantasy System. 

An interactive story is a first-person experience within a fantasy world, in 
which the user may create, enact, and observe a character whose choices 
and actions affect the course of events just as they might in a play. The 
structure of the system ... enables first-person participation of the user in 
the development of the story or plot, and orchestrates system-controlled 
events and characters so as to move the action forward in a dramatically 
interesting way [Laurel, 1986]. 

Her definition describes very well what an interactive story experience should be like and 
is a starting point for designing an interactive story system. Laurel states that the system 
must “move the action forward in a dramatically interesting way.” This implies the story 
is not being driven to a single predetermined ending; rather the climax and ending are 
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free to ehange as the story progresses. The system strives to present the best story that 
emerges based on the user interaction, as opposed to presenting a specific story. 

2. Interactive Story 

For the purposes of discussion, it is assumed that a fully defined and functional 
“fantasy world” exists. The existence of this world implies that there are laws and rules 
in place to govern the interactions and events in this world. The world is populated with 
autonomous computer controlled characters, driven by internal goals and cognizant of the 
laws and rules of the world. The autonomous characters are free to explore and 
manipulate the environment, again subject to the rules of the world. The interactive user 
is a participating character in the world. Like the autonomous characters, the user’s 
character is free to roam and interact in the story world. The difference is that the goals, 
desires and actions of the user’s character are influenced not only by the environment and 
interactions with the other characters, but also directly by the user. 

The interaction of the user impacts the development of the story, but not at 
discrete points in an obtrusive way. Rather, the user’s interaction serves to mold the plot 
in a continuous fashion. Michael Mateas and Andrew Stern describe the evolution of an 
interactive story plot in their paper Towards Integrating Plot and Character for 
Interactive Drama. They write: 

Changes in the plot should not be traceable to distinct branch points; the 
player will not be offered an occasional small number of obvious choices 
that force the plot in a different direction. Rather, the plot should be 
smoothly mutable, varying in response to some global state which is itself 
a function of the many small actions performed by the player throughout 
the experience [Mateas and Stem, 2000]. 

As the user interacts with the story world, the events and characters in the world 
are influenced so as to move the action forward in an interesting way. The influence can 
be as subtle as a slight adjustment to a character’s personality to encourage a certain type 
of behavior, or as overt as overriding their goals to force an action that is in the best 
interest of the overall story. Obviously the subtle approach is preferred. 

The challenges of presenting a compelling story in the face of interactivity are 
well documented [Bates, 1992], [Murray, 1998], [Weyhrauch, 1997], [Mateas and Stem 
2000], [Szilas, 2001]. A good story is well planned and scripted. The author goes to 
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great lengths to ensure events unfold in a sequence that best captures the author’s desires 
and is interesting to the audience. In the case of an interactive story, the task of writing 
and presenting the story is more complicated because the audience is one of the 
characters in the story (the interactive user). In which case, the author must forfeit some 
measure of control over how the story plays out in exchange for interactivity. This 
balance between author control and interactive freedom for the user is at the heart of 
developing an interactive story world capable of producing a large number of diverse 
story lines. 

In the model of interactive story presented in this dissertation, the author exerts a 
high level of control over developing the story world, but forfeits direct control over the 
events that occur (Figure 1). The story world is defined through laws and rules that 
govern character-to-character and character-to-environment interactions. Included in this 
definition of the world are the autonomous story characters with their own goals and 
personalities. While direct control over the events is limited, the temporal relationship 
between events is established indirectly through the laws and rules imposed by the author 
on the story world. 
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User Interaction 


Laws - Indirect Control 


Autonomous Characters 




- Environment (lakes, trees, etc) 


Figure 1, Story World Diagram 

The story characters, including the character being influenced by the user, are 
turned loose in the story world. The result is a story line that results from the interaction 
of the characters with each other, the environment and the user. It is conceivable that a 
single story world could yield many different story lines. 

The character’s goals serve to focus the interactions so as to produce an 
“interesting” story, where “interesting” is relative to the story world domain and the 
objectives of the author. If the system is designed for war-game scenarios, “interesting” 
may be defined in terms of the feasibility and uniqueness of the scenarios and how well 
they meet the training requirements. If the desire is to present an action-adventure story, 
“interesting” may be defined in more classical narrative terms employed by screenplay 
writers and authors of fictional works. 
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3, Summary 

Interactive story concerns itself with building simulated worlds populated with 
autonomous characters and entities, within which the user participates in the story or 
simulation. A significant amount of research has taken place in the area of building 
believable autonomous characters to populate the story world [Reilly, 1996], [Loyall 
1997], [Mateas, 1997], [Blumberg, 1996], [Hayes-Roth et al, 1996] and [Rickel et ah, 
2001]. However, the body of work on developing systems where the story lines and 
scenarios adjust to the autonomous behavior of individual agents is smaller in scope 
[Weyhrauch, 1997], [Marsella et al, 2000], [Mateas and Stern, 2000] and [Swartout et 
a/., 2001]. 

The goal of interactive simulation, whether it is a virtual story or a combat 
simulation, is to present the user with an experience that suspends their disbelief in the 
artificialities imposed by the system. In this way, the user feels it is a “real” experience. 
From the DoD perspective, this results in more realistic and effective training, as well as 
more accurate assessments of the systems, tactics or doctrine being evaluated. 

The entertainment industry has long known that to achieve this suspension of 
disbelief, it is not sufficient to simply produce a technically accurate simulation. It is the 
unfolding of events and presentation of the story, along with rich believable characters, 
that makes for a truly effective and immersive experience. 

D, APPROACH 

This research formally describes the architecture of MAISE (Multi-Agent 
Interactive Story Engine) i, a simulation engine that uses a multi-agent system (MAS) to 
produce interactive stories. A domain specific story world is constructed from domain 
independent agent-based modeling constructs. The constructs of tickets and connectors 
[Hiles et al, 2001] are combined with scenes and interactions to construct dynamic plans 
that manifest themselves as goal-directed stories. The intention is not to tell a specific 
story, but to let the story unfold within the bounds of the story world based on the agents’ 
interactions with one another and the user’s interaction with the system. A typical story 
consists of goal driven autonomous characters, props, constraints, and a collection of 

1 For the remainder of the dissertation, MAISE is referred to simply as the story engine. 
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potential scenes, along with media, dialog, and character interactions to populate the 
scenes. These story elements are combined dynamically at run-time to generate a story 
line that adapts to the user’s interventions and is closely aligned to the goals of the 
protagonist. 

In this architecture, story lines are a by-product of observing constraints defined 
on classes of agents within the simulation. Thus, story lines are plans generated through 
discovery rather than following fixed plans defined beforehand. The story lines are 
generated through a simulation process called connecting, whereby agents are bound 
together according to a “best-fit” axiom. The by-product of the connection is the next 
scene in the story (or step in the plan). In this way, dynamic story lines, or plans, are 
evolved as the simulation runs. The bottom-up, MAS design of the story engine has 
resulted in a simulation engine that is domain independent and can be scaled to 
accommodate stories of any breadth and depth. 

The story engine is an instance of a larger family of simulations entitled 
Connector-based Multi-Agent System simulations (CMAS). This dissertation formally 
presents the CMAS architecture, followed by the specific description as it applies to the 
story engine. 

Finally, the story engine was fielded as the underlying simulation engine behind 
the U. S. Army’s America’s Army: Soldiers (AA: Soldiers) project. The instantiation of 
the story engine, as it applies to AA: Soldiers, is presented. As a component of ^4^4.- 
Soldiers, the story engine is an integral module in an Interactive Story Generation System 
(ISGS) that is patterned on the model-view-controller architecture (Figure 2). The story 
engine encompasses the model and controller, while the Scene Rendering Subsystem 
provides the view. Interactive access to the story world, via the story engine, is provided 
by the graphical interface. Development of the Scene Rendering Subsystem and 
graphical interface is outside the scope of this dissertation, however, they are described 
briefly in Chapter VI. 
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Figure 2. Interactive Story Generation System 
E. CONTRIBUTIONS 

This dissertation provides a fundamental new approach to generating interactive 
stories. The following are the specific contributions of this research; 

• A formal description of a general-purpose multi-agent system architecture, 
called a Connector-based Multi-Agent System (CMAS). 

• A domain independent, scalable simulation architecture, based on the CMAS 
architecture, for generating logically connected and goal-directed interactive 
stories (Story Engine CMAS). 

• A bottom-up simulation methodology to discover novel and unexpected plans 
for achieving an agent’s goal(s). 
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• A formal description of an agent communication, coordination and control 
process based on connectors and connecting. 

F. DISSERTATION ORGANIZATION 

The remainder of this dissertation is organized as follows: 

• Chapter II. Related Work, This chapter is provided to give the reader a 
background on research conducted in the area of interactive narrative and 
multi-agent simulation planning systems. The work that is most relevant to 
this dissertation is described in detail. 

• Chapter III, Multi-Agent Systems, Multi-agent systems are defined and a 
working framework describing multi-agent systems is presented. This 
framework is based on work by Jaques Ferber [Ferber, 1999]. 

• Chapter IV. Multi-Agent System Research. A survey of MOVES Institute 
computer generated autonomy research is presented along with a formal 
definition of connectors and tickets. The research presented in this chapter is 
fundamental to the development of the interactive story engine. 

• Chapter V. Connector-hased Multi-Agent System Architecture. This 
chapter brings together tickets, connectors and multi-agent systems to 
formulate a formal definition of a Connector-based Multi-Agent System 
(CMAS) architecture. A simulation model is developed based on the process 
of connecting and the agent interaction that occurs during a connection. 

• Chapter VI. Interactive Story Generation System. The architecture of the 
story engine is presented in terms of the formal description of a Connector- 
based Multi-Agent System (CMAS). The story engine is combined with a 
real-time movie animator and generative text-to-voice system to create an 
Interactive Story Generation System (ISGS). 

• Chapter VII. CMAS Data Set Implementation - America’s Army: 
Soldiers. America’s Army is a suite of two PC based computer games 
developed for the U.S. Army intended to provide young people with accurate, 
easy-to-assimilate information about the Army. America’s Army: Soldiers is a 
story-based role-playing game where a player guides a character through an 
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Army career. This section describes the tiekets, eonneetors, seenes and 
characters, ineluding their goals and personalities used to ereate the story 
world. America's Army: Soldiers is presented as a proof-of-eoneept of this 
dissertation research. 

• Chapter VIII. Conclusions and Recommendations. The dissertation 
concludes with a summary of eontributions and suggested applieations for the 
story engine. A number of avenues for follow-on work and extensions are 
also provided. 
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II. RELATED WORK 


A, INTRODUCTION 

Stories have long played a eentral role in human eulture. Learning, 
eommunication, soeial interaction, arts and even recreation all center around stories. 
Before man could write, stories were used as a means of passing information from one 
generation to the next. It was much easier for scholars to pass along important findings 
and key events in the culture’s history when they were organized and relayed as a story. 
Along with being used as a means of exchanging information, stories provide a 
framework for how we approach the world. They provide meaning to the reams of data 
that flood our senses every day. Through the use of stories, whether consciously or sub¬ 
consciously, we make sense of the world. We order its events and find meaning in them 
by assimilating them into familiar narratives. 

The media for relating stories has traditionally taken on familiar forms such as 
written work (rock wall carvings, ancient scrolls, books, magazines and newspapers), live 
performance (story-telling, theatrical plays and musicals) and cinema. These forms all 
have one thing in common; they are intended to tell a story to a non-participating 
audience. The stories are not written with the intention of changing the plot as the story 
progresses based on the desires of the audience. From the moment the first scene opens, 
every act, every action, and every line of dialog is scripted to achieve the specific goals of 
the author. There are no mechanisms for the audience to influence the story as it is being 
told. Why not? Because it is very difficult from the literary perspective. How does one 
tell or write a good story if they don’t know what the characters are going to do or how 
they are going to react, particularly when it is one of the main characters (the interactive 
user). 

This chapter examines alternative approaches that have been used in the 
interactive drama domain and related domains. The evolution of interactive 
entertainment and story understanding and telling systems are also presented from the 
perspective of their influence on current approaches to interactive narrative. Along the 
way, those efforts that best define the current state of research in the field of interactive 
story generation have been expanded upon. Finally, this chapter lays the groundwork for 
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a multi-agent system simulation approaeh to generating stories by examining two agent- 
based planning systems. 

B, NARRATIVE INTELLIGENCE 

Narrative Intelligence is a term that was coined by Marc Davis and Michael 
Travers of MIT’s Media Lab in 1990 to describe a field of study combining artificial 
intelligence and literary theory [Davis and Travers, 1997]. Their collaboration was the 
foundation of a group of students and faculty interested in pursuing interdisciplinary 
work at the intersection of artificial intelligence, literary theory, media studies, cognitive 
science and human-computer interaction design. Drawing on a diverse range of 
influences, the researchers have explored a wide variety of topics relevant to narrative 
intelligence. A number of common themes have emerged including; narrative as the 
basis for human-computer interface design [Laurel, 1991], intelligent agents using 
narrative structure to model aspects of human intelligence [Schank, 1990], story¬ 
understanding systems, story-telling systems, interactive entertainment, and interactive 
drama. 

While the term Narrative Intelligence emerged in 1990, early foundations of this 
area of study are grounded in the field of artificial intelligence (AI). 

1, Artificial Intelligence Story Systems 

In the 1970’s and early 1980’s, there was a substantial amount of interest in story 
understanding and generation. A research group at Yale, headed by Roger Schank, 
explored the use of scripts, plans and goals to understand narratives [Schank and 
Abelson, 1977]. Scripts and plans are a means for achieving goals. Scripts are used to 
capture the notion of a stereotyped situation and the sequence of actions appropriate for 
the situation. They are prepackaged sets of expectations, inferences, and knowledge that 
are applied in common situations. For example, in a restaurant script, the sequence of 
actions of entering, finding a seat, ordering a meal from a server, the meal being prepared 
and served, eating, paying the bill and leaving the restaurant would constitute a script. 
Scripts provide rules for understanding the connectivity in a stereotypical situation. 

Since it is not possible to define a script for every possible situation, the notion of 
a “plan” was developed to deal with situations not previously encountered. Plans are a 
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repository of general information and provide a means of eonneeting events that eannot 
be eonneeted by the use of an available seript. They provide the meehanism for 
understanding events about whieh there is no speeifie information. 

The Yale group, as part of their work, developed a series of programs to help 
understand textual narratives. These programs ineluded SAM (Seript Applier 
Meehanism), TAIL-SPIN [Meehan, 1976], and PAM (Plan Applier Meehanism). These 
early systems were intensely knowledge based, funetioning in very limited domains. To 
make them more general, extensive knowledge engineering would be required. In 
addition, the plans were statie; bound before the story ever began. Diseussion of the 
speeifies of eaeh program is presented in the following seetions on story-understanding 
and story-telling. 

2, Story-Understanding Systems 

Story-understanding systems are systems designed to take as input a narrative 
diseourse, and provide as output information about the story. The information is 
normally shared as part of a two-way question and answer dialog between the user and 
the software program. The goal is for the program to not only reeite faets direetly from 
the story, but also be able to provide the user with information that eould be logieally 
inferred from the story events. 

SAM was a program written at Yale designed to understand stories that rely 
heavily on seripts. To understand the stories, SAM ereated a linked eausal ehain of what 
took plaee in eaeh story. A seript applier then made inferenees about events that must 
have oeeurred between events speeifieally mentioned. This resulted in a large 
“eoneeptual dependeney network.” From this, the system was able to paraphrase the 
original story, expand the story with the inferred events, or summarize the story based on 
measures of the relative importanee of events. The network eould also be queried to 
answer questions about the story. 

PAM was another story-understanding system, but unlike SAM, whieh was based 
on seripts, PAM was based on plans. PAM used knowledge about goals and plans to 
diseern the intentions of the eharaeters in the story. The program kept traek of the goals 
of eaeh eharaeter and interpreted their aetions as means of aehieving those goals. 
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3. Story-Telling Systems 

Unlike story-understanding programs, story-telling systems make no attempt at 
understanding the motivation behind the events and aetions of the eharacters nor making 
inferenees about why the events oeeurred. TALE-SPIN is an example of an early story¬ 
telling program written by Jim Meehan while a student at Yale in the mid-1970’s 
[Meehan, 1976]. TALE-SPIN made up non-interaetive stories about animals by 
simulating a world, assigning goals to the eharaeters, and deseribing what oeeurred when 
these eharaeters interaeted with other eharacters and objects in the environment in the 
pursuit of their goals. It was used as a means of testing the goal and planning apparatus 
proposed by Schank’s group. TAEE-SPIN worked by first defining, in great detail, how 
a character might plan to accomplish a goal. The system required explicit plans for every 
possible goal a character might wish to achieve. Then, given a set of characters and a 
starting goal, the program could generate stories using its basic knowledge of the 
elements needed to construct a story, the elements of planning and achieving goals, 
knowledge of language structure, as well as general story world knowledge. 

AI planning techniques are still being explored today as models for narrative 
plots. Michael Young, lead researcher at North Carolina State University’s Eiquid 
Narrative Group is extending recent techniques in AI planning, including causal and 
hierarchical structures of plans, to capture the key features of narrative structure [Young, 
1999], [Young, 2000]. 

C. NARRATIVE THEORY AND SCREENPLAY STRUCTURES 

The breadth and depth of literature on narrative theory, dating back to 330 BC 
[Aristotle, 330 BC], is sufficiently exhaustive so as to prevent a thorough review here. 
However, a brief, high-level description of story structure is presented. 

Story structure is often described as a dramatic arc where the vertical axis 
represents tension and the horizontal axis is time (figure 3). As the story progresses, 
tension builds and falls based on incidents and interactions in the story. All the while the 
overall slope of the arc shows an increase in tension. This continues until some crisis 
occurs resulting in the climax. During the climax, questions are answered and the 
tensions are resolved. 
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Tension 



Figure 3, Dramatic Structure 

Recent examinations of the movie industry and screenplay writing have produced 
a variety of multi-act structural models for representing dramatic arc. These models fit 
well with the task of generating interactive stories, where the user’s view is more closely 
aligned with interacting in a movie or play than it is with reading a novel. 

In David Siegel’s essay The Nine-Act Structure [Siegel, 2001], he describes a 
general model for a movie’s structure based on either seven or nine acts (Figure 4). Flis 
comparative analysis between his model and a number of recent movies indicates that a 
high percentage of top grossing films have followed the nine-act structure. 


GOOD FORTUNE 


_6_BAD FORTUNE 

FILM TIME 


THE NINE-ACT STRUCTURE 


ACT 0: SOMEONE TOILS LATE INTO THE NIGHT. 
ACT 1: START WITH AN IMAGE. 

ACT 2: SOMETHING BAD HAPPENS. 

ACT 3: MEET THE HERO. 

ACT 4: COMMITMENT. 

ACT 5: GO FOR THE WRONG GOAL. 

ACT 6: THE REVERSAL. 

ACT 7: GO FOR THE NEW GOAL. 

ACT 8: WRAP IT UP. 



Figure 4, David Siegel’s Nine-Act Structure from [Siegel, 2001] 


17 









At a more coarse level of granularity, Kristin Thompson presents a four-act 
structure for movie making [Thompson, 1999]. The acts include setup, complicating 
action, development, and climax. In setup, the initial situation is presented. This is often 
where the protagonist establishes one or more goals. The complicating action takes the 
action in a new direction. This may involve the protagonist pursuing a previously 
established goal in a new way, or possibly being faced with an entirely new goal or 
obstacle. Development is recognized as the phase of the story where the protagonist 
works toward the achievement of their primary goal(s). As the climax portion begins, the 
action shifts to straightforward progress toward a final resolution. 

Stuart Voytilla cites 50 major films produced over the past 70 years as evidence 
of Christopher Vogler’s movie storytelling model based on the protagonist’s journey 
through 12 stages of a story [Voytilla, 1999]. Metaphorically speaking, the 12 stages 
take the protagonist from their everyday life, to the acceptance of some challenge, 
through the struggles for achievement and finally back home. Vogler’s “Hero’s Journey 
Model” is depicted in Figure 5. 

In spite of their apparent differences, each of these models is functionally 
equivalent, with the primary difference being the level of fidelity of the model. The 
twelve phases of the hero’s journey relate directly to the four acts identified by 
Thompson. Likewise, Siegel’s nine-act structure can be mapped onto a four-act 
structure. Additionally, these models share a common link between the stages of the 
story and the central character’s goals. In each case, transitions from one stage to the 
next are initiated by a goal shift on the part of the protagonist or another key character. 
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Act I 

Separation 


Act III 
Return 




G 






Ordinary I World 



Call to Adventure 



Tests, Allies, 
and Enemies 


Act II - A 
Descent 


Approach 
the Inmost Cave 


Act II - B 
Initiation 


Figure 5, Vogler’s Hero’s Journey Model from [Voytilla, 1999] 

D, INTERACTIVE ENTERTAINMENT 

Interactive entertainment is a very general notion and determining what 
constitutes interactive entertainment is subjective. For instance, all of the following can 
probably be considered a form of interactive entertainment: participating in an academic 
debate, coming to bat in a baseball game, and riding a magic carpet in Disney’s virtual 
theme ride Aladdin [Pausch et al, 1996]. For the purposes of this dissertation, the 
discussion is limited to interactive entertainment in the computer-based domain. But 
even this encompasses a broad range of applications, including early adventure games, 
hypertext stories, video games, multi-user dungeons (MUDs) [Weyhrauch, 1997] and 
networked virtual environments, just to name a few. 
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Early forms of computer-based interaetive entertainment took the form of text- 
based adventure games. In games sueh as Zork [Lebling et al, 1979], the user aets as an 
explorer and solves puzzles to gain additional information and achieve some end goal. 
The user would enter eommands at the keyboard and the system responded with text 
output deseribing what the user eould see, hear and feel. This game presented an 
immersive text-based setting with its own eonsistent reality, but there was no real plot. 
Most of the stimulation eame from solving the puzzles. The user was provided with a 
sense of inereasing antieipation as the level of diffieulty inereased. But in the end, these 
systems did not tell a story, nor was telling a story of primary eoncern to the developers. 
The experienee was interaetive in that the user’s choiees affected the events of the game, 
but the user did not really interact with the characters. The souree of interaetion was the 
ehallenge of navigating through an unseen maze, in search of a single exit point. Paths 
through the game either led to dead ends or to a single prized ending. These programs 
were engaging in the mental ehallenge they presented their audienee, but they provided 
little in the way of plot. 

Hypertext stories are somewhat similar to early adventure games in that the reader 
is presented a maze they must navigate. Though in the ease of hypertext stories, it is a 
maze of story pieees. Hypertext trades some of the interaetivity offered by the adventure 
games for a mueh stronger story structure. The reader follows a path through the network 
of story pieees assembled by the author. The interaction occurs as a result of the user 
selecting from a small number of ehoiees at distinet branch points provided by the author 
via hyper-links. Between eaeh of the hyper-link braneh points, the reading of the text is a 
linear experienee. While the user’s ehoiees dietate the outeome of the story, it is the 
author who eontrols the flow. The network of possible stories is established at eompile 
time and is statie, resulting in a small finite number of paths or stories. Compare this to a 
oolleetion of books where eaeh book is based on a unique path through the network of the 
story pieces. [Weyhrauch, 1997] deseribes a path through a hypertext story as analogous 
to reading a single book selected from the eollection. 

Hypertext provides a solid story experienee, but falls short of adventure games in 
its level of interaetivity. By providing the reader a small number of ehoiees intermixed 
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between long periods of linear text, the reader rarely feels as though they are partieipating 
in the story. The goal of interactive story is to immerse the user in the story world and 
make them feel as though they are an integral part of the action. 

Video games are an example of interactive entertainment that are high on 
interactivity and sensory input but low on story. Playing these games can be a very 
powerful experience particularly with the advances in computer graphics. These systems 
appear more and more lifelike with the introduction of each new generation of hardware, 
providing a powerful sense of immersion. Soon, a computer-generated scene will be 
comparable to a movie scene, at least from a visual sense. While visuals are extremely 
important, they are only part of the experience. On the whole, video games lack 
meaningful character interaction and have little or no story structure. Even when there is 
an underlying story, it is merely to set the stage for the action and has little use 
throughout the remainder of the game. Video games rely on visual, aural and haptic 
stimulation, along with highly interactive action to draw the user into the game. An 
interactive story system strives to capture the immersive qualities of video games while 
providing meaning to the action. 

Finally, the interactive worlds of Multi-User Dungeons (MUDs) allow distant 
players on the Internet to share a common virtual space in which they can chat with one 
another. Players improvise scenes together and imagine fictional worlds. MUDs are 
environments for fantasy play that allow people to create and sustain elaborate fictional 
personas over long periods of time. They are truly interactive in that they are meant to be 
experienced in the first person, not to be viewed or watched by an outside audience. 
However, MUDs fail as an interactive story because there is no consistent story behind 
the interactions. It is much like improvisational acting where characters (users in role as 
their MUD persona) explore the environment, interacting with one another in a manner 
consistent with their fantasy role. 

While interactive entertainment can be highly immersive, it tends to regard 
interactivity and story as separate and almost incompatible. Interactive drama strives to 
meld the two, whereby the story being presented is inextricably linked to the actions of 
the user. 
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E, INTERACTIVE STORIES 

Previous efforts at developing an interaetive story system have met with limited 
sueeess. Brenda Laurel [Laurel, 1986] proposed a design for an interaetive fantasy 
system, identifying the system eomponents and neeessary funetionality, but a system was 
never developed. More reeent work as part of Carnegie Mellon University’s OZ Projeet 
introdueed the notion of a plot graph, a partial ordering of plot events [Kelso et al., 1993] 
[Weyhraueh, 1997]. Weyhraueh used the plot graph as a foundation for searehing all 
possible orderings of plot events in order to eonstruet a story. While this approaeh 
sueeessfully showed that it was possible to build an interaetive story, it did not seale well 
to more eomplieated stories. The University of Southern California Institute for Creative 
Teehnologies (ICT) has developed a network-based approaeh to produeing interaetive 
training seenarios [Swartout et al., 2001]. ICT’s Mission Rehearsal Exereise (MRE) 
system allows users to experienee a story in a highly immersive environment. However, 
user interaetion points are fixed and tightly eonstrained in order to ensure the story 
eonforms elosely to the training objeetives of the seenario. A more detailed deseription 
of these works is presented later in this ehapter. 

Other approaehes to eomputer-generated stories have explored the use of story 
pieees in the form of standalone audio, video or text elips, to tell a story. Kevin Brooks 
of the Interaetive Cinema Group at MIT’s Media Lab uses a single set of predefined story 
pieees to tell many different linear stories [Brooks, 1999]. He uses story agents to 
arrange the pieees, resulting in stories that take on the personality of the respeetive story 
agent. 

Developing a software program to tell a pre-seripted linear story might present 

some artistie ehallenges, but in the end it would be a straightforward effort. This is 

beeause the software engine driving the story has eomplete knowledge of the seript and 

ean exereise total eontrol over the environment and the eharaeters. When interaetivity is 

introdueed into the equation, the software no longer has eontrol over the story. One of 

the eharaeters (the user) is now free to aet based on what has previously oeeurred, their 

interpretation of these previous events, and their pereeptions of where the story might be 

headed. In spite of this, the system hasn’t neeessarily forsaken all eontrol. It ean limit 

what the user sees and eonstrain the user’s options so as to nudge the aetion in a 
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particular direction. So the introduetion of interaetivity requires a tradeoff in narrative 
eontrol. At one extreme there is unbounded interactivity. The danger here is that the user 
ean be presented with so many options that they flounder aimlessly, never moving 
forward with a eoherent story. Alternatively, if the user is provided no flexibility, the 
experience is no different than watehing a movie or reading a book. In designing an 
arehitecture to control the flow of action in an interactive story, a balanee must be struck 
between interaetivity and narrative control. 

In his dissertation, Peter Weyhraueh deseribes four approaehes to guiding the 
action in an interactive story system [Weyhraueh, 1997]. The first is to build an 
environment with a great deal of interaetivity and aetion, but no drama eomponent at all. 
The idea is that if there are enough things going on, then something interesting is bound 
to happen. MUDs tend to fall into this eategory. A second approach is the use of an 
implicit dramatic structure. The strueture follows from the eharaeters and their goals. 
Video games and military simulations are examples of this approaeh. In a military battle 
simulation, the two sides have goals of eonquering eaeh other; this provides an implieit 
structure to the action. In these first two approaehes, there is no real attempt to impart a 
narrative strueture onto the aetion and interaetions. If a story emerges, it is a by-produet 
of the action. 

The third approaeh Weyhraueh deseribes is a fixed story sequenee. The user has 
some level of freedom, but there is a eentral direetor that ensures fixed story events 
oeeur. With this approach, narrative strueture takes precedenee over interaetivity. 
Tinsley Galyean describes this as “narrative guiding interaetivity” [Galyean, 1995]. The 
fourth approaeh is to use an explicit drama manager to guide the story events. The drama 
manager provides input to the eharaeters and adjusts the story environment to encourage 
the action to follow a narrative structure. 

These last two approaches to guiding aetion lead to two general eategories of 
work in interaetive narrative. The first is aimed at ereating and presenting a specifie 
story. That is, the user experienees the story the author has written. While it is possible 
and desirable for the plot to adjust and reaet to the variations resulting from the user’s 
interaction, in the end, the author’s story is told. The variations are constrained to ensure 
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that critical plot points are met. The seeond eategory of work foeuses on dynamieally 
building a plot based on the user’s interaction. In some sense, the user builds their own 
story. The events and actions that occur are eonstrained both by the realm of possibility 
of the story world and an overriding, high-level narrative strueture. It is the function of 
the drama manager to impart the narrative structure on the events of the story world. 

1, Centralized Drama Manager 

The first formal deseription of an interaetive story system based upon a 
eentralized drama manager was presented by Brenda Laurel [Laurel, 1986]. Laurel 
proposed an “interaetive fantasy” system whose primary eomponents were eomprised of 
a eentralized drama manager, a world model (or story world), system eharaeters with a 
rieh set of personality traits, and an intelligent user interfaee into the story world. At the 
heart of the system was a rule-based drama manager ealled PLAYWRIGHT. It was 
playwright’s job to give eaeh character their formal speeifieations for the next 
ineident. In turn, the characters would provide their suggestions for action back to 
PLAYWRIGHT that would implement the first aeceptable suggestion. While Laurel’s 
system was a theoretieal design and never implemented, it was the first thorough 
treatment of the issues involved with developing an interaetive story system. 

A researeh group at Carnegie Mellon University has an ongoing projeet to 
develop an interaetive drama system that seamlessly eombines believable agents with an 
interaetive story strueture [Bates, 1992], [Kelso et al, 1993], [Mateas, 1997] and [Mateas 
and Stern, 2000]. This group, known as the Oz group, works with the overriding 
philosophy that drama is a eombination of story, characters and presentation. They focus 
on building worlds that give equal attention to believable agents, interactive plot and 
presentation. Fundamental to their notion of an interaetive plot is the use of a eentral 
drama manager. Figure 6 shows the high-level arehiteeture of the Oz projeet taken from 
[Mateas, 1997]. The world eontains eharaeters that exhibit rieh personalities, emotion, 
soeial behavior, motivations and goals. The user interaets with this world through a 
presentation medium. Finally, the drama manager guides the experienee of the user and 
makes things happen. 


24 




Figure 6, Oz Project Architecture from [Mateas, 1997] 

Central to the idea of interaetive stories is the idea of believable eharacters. This 
section will touch upon the work with believable agents that populate the world, but will 
focus primarily on the drama manager. A more thorough treatment of believable agents 
can be found in [Loyall, 1997]. Likewise, much of the Oz project work on presentation 
deals with generating English narrative [Bates and Kantrowitz, 1992] and is not discussed 
here. 

a. Believable Agents 

First, the concept of “believable character” must be established. A 

believable character is one who seems lifelike, whose actions make sense and who allows 

the user to suspend disbelief. Note that nowhere in the description is anything said about 

realism. For the purposes of drama and interactive stories, the character is believable if 

he or she remains in role and exhibits consistency of action, motivation and thought. 

Consistency is key to making a character believable. As described in [Mateas, 1997]; 

A character may be smart or dumb, well adapted to its environment or 
poorly adapted. But regardless of how “smart” a character is at dealing 
with their environment, everything they do, they do in their own personal 
style. 

Once a character is developed in a story, the user forms certain 
expectations as to how that character will behave and react. The character must follow 
these expectations to remain believable in the eyes of the user. An “out of character” 
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response or unwarranted aetion serves to break the users suspension of disbelief. 
Believable eharaeters are just that, believable, they are not neeessarily realistie. 

Bryan Loyall of the Oz group defined the following set of requirements 
for believable agents [Loyall, 1997]; 

• Personality - Personality deseribes the unique and speeifie things 
about the eharaeter and is what makes the eharaeter interesting. 

• Emotion - Charaeters must appear to have emotional reaetions and 
have a means of expressing the emotions. 

• Self Motivation - The agents must not only reaet to external stimuli, 
but also engage in aetions on their own aeeord. They must be able to 
aet based on internal drives and desires. 

• Change - Charaeters must grow and ehange over time eonsistent with 
their personality. 

• Soeial relationships - Charaeters interaet with one another and these 
interaetions are infiueneed by whatever relationships the eharaeters 
have with one another. 

• Consisteney of Expression - To be believable, every avenue of 
expression available to the agent must eonvey a unified message 
eonsistent with the agent’s personality and emotion. 

• Illusion of life - This is a eolleetion of requirements that ineludes 
pursuing multiple goals, eapable of parallel aetion (e.g., walk and talk 
at the same time), possessing eapabilities sueh as movement and 
pereeption, and finally the ability to reaet to stimuli in the 
environment. 

These requirements led to the development of an agent arehiteeture for 
believable agents ealled Hap [Loyall, 1997], [Bates, Loyall and Reilly, 1992], [Loyall 
and Bates, 1991]. Hap is both a language for defining agents and a software engine for 
eontrolling the agents. The language allows the author to define goals, behaviors and 
aetions. At the heart of the arehiteeture is an active behavior tree (ABT). The ABT is 
the main proeessing data strueture in a Hap agent. Hap exeeutes by repeatedly 
performing a multi-step loop that proeesses the ABT, expanding or shrinking the tree 
based on aetions that have been eompleted, ehanges in the world and goals that have been 
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suspended. Onee the tree is updated, Hap ehooses the next aetion for the agent from the 
set of available aetions in the ABT. The seleetion is based upon an internalized priority 
seheme with preferenee given to working on the same goals and behaviors that the agent 
had been reeently pursuing. This priority seheme serves to maintain a sense of foeused 
purpose to the agent’s aetivity. 

The follow-on to Hap, ABL (A Behavioral Language), is being used as the 
behavioral model for the believable agents in an interaetive drama projeet entitled 
Fa 9 ade. ABL elosely follows the Hap semanties. A detailed deseription of ABL and the 
differenees ABL and Hap ean be found in [Mateas and Stern, 2002]. 

b. Drama Manager 

The purpose of the Oz drama manager is to sequenee major story events 
(plot points) into a eoherent story line, while taking into aeeount the aetions of the 
interaetive user, thereby aehieving an interaetive plot. The Moe arehiteeture for dramatie 
guidanee was developed for this purpose [Weyhraueh, 1997]. Moe funetions by 
eontrolling the sequeneing of major events (plot points) throughout the story. Moe uses 
an adversarial seareh teehnique, not unlike those used by ehess games, to generate 
sequenees of plot points. It then applies an evaluation funetion to rate eaeh sequenee. 
The highest rated sequenee is used to seleet the next event in the story. 

A plot sequenee is made up of Moe moves and user moves. User moves 
are the aetions that the interaetive user ean take in the story. For example, in his 
dissertation [Weyhraueh, 1997], Weyhraueh uses a mystery story to demonstrate the Moe 
arehiteeture. Examples of possible user moves would be to seareh the murder seene, 
diseover a pieee of evidenee or eonfront a suspeet. The set of all possible user moves are 
formed into a plot graph. A plot graph is a partial ordering of story events linked together 
in a direeted aeyelie graph. The plot graph provides a basie hierarehieal strueture to the 
events. For instanee, in the plot graph, the event of sending evidenee to the erime lab for 
analysis eomes after the event of finding the evidenee. The plot graph is used for 
generating a set of legal user moves from whieh to seleet the next move. The aetual 
seleetion oeeurs though the adversarial seareh aeeording to an evaluation funetion. 

Moe moves are those aetions the drama manager ean take to guide the 

story and move it in a desired direetion. An example of a Moe move might be to ereate a 
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commotion so as to attract the user’s attention so they stumble across a piece of evidence. 
The intertwining of the Moe moves with the user moves results in a eompleted plot 
sequence or scenario. 

Weyhrauch developed an evaluation function to rate scenarios based on a 
weighted linear combination of the following seven faetors: 

• Thought Flow - A measure of whether one user event relates logieally 
to the next event. 

• Aetivity Flow - A measure of how bored the user might feel by 
moving around with nothing oecurring. 

• Options - A measure of how mueh freedom of ehoice the user 
perceives they have. 

• Motivation - Measures whether the user’s aetions are related to their 
active goals. 

• Momentum - Certain events make sense if they oceur in close 
proximity to one another. Momentum provides a measure for this. 
The story’s author determines which events are related and how elose 
in time they should oceur. 

• Intensity - Measures the building of excitement as the story 
progresses. This eaptures the elassieal dramatic arc discussed earlier 
in the ehapter. 

• Manipulation - A measure of how manipulated the user feels by the 
drama manager. 

It is important to note that the evaluation function only works on complete 
seenarios. Logieally, a scenario is made up of two parts; the first consists of the fixed 
sequence of events that have already occurred and the seeond is some sequeneing of the 
events that have yet to oeeur. Beginning with the sequence of events that have already 
oceurred, an adversarial seareh teehnique is used to explore all possible sequenees of 
events yet to occur (Moe and user moves). The search tree is expanded by eonsidering 
all possible Moe moves and legal user moves (the plot graph is used to determine which 
user moves are legal). The expansion continues until every user move has been used. 
Eaeh of these eompleted scenarios is evaluated, the best one seleeted, and this determines 
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the next move in the seenario. The proeess then starts all over again. This type of 
algorithm results in a tree with a tremendously high branehing factor. Even with a small 
number of user moves and Moe moves, the problem of searching the entire tree is 
intractable. For example, with 15 user moves and 15 Moe moves, there can be as many 
as 30! (2.65x10 ) scenarios that must be initially evaluated. As the story progresses and 
the history of events becomes fixed, the complexity of the problem decreases, but not fast 
enough to make it manageable. As a result, Weyhrauch developed two heuristics for 
rating scenarios in real time. The details of the heuristics are not described here but can 
be found in [Weyhrauch, 1997]. 

Weyhrauch’s work showed that an interesting and dramatically appealing 
story can be presented while remaining true to the user’s interactive freedom. However, 
the plot graph paradigm and use of an adversarial search with user and Moe moves does 
not scale well. The architecture is quickly overcome by stories involving any sizeable 
number of plot events. Additionally, his work required that every possible event in the 
story be defined in advance and that every one of these events be used in the final 
scenario. 

Using the Moe system, every story that occurs will consist of the same 
basic events, the only difference being how the events come about and in what order. 
The ending is always the same, only the path to that ending is variable. It is this 
variability that is at the heart of the interactive nature of his architecture, but in the end, 
all paths must lead to the capture of the murderer. 

The multi-agent system based story world approach described in the 
previous chapter is intended to present an interesting story, given the characters and 
constraints of the story world, not a specific story. In the case of a mystery, there are 
many good stories that can be told, not all of which end with the perpetrator being caught. 
For example in the 1999 film Entrapment with Sean Connery and Catherine Zeta-Jones, 
Connery plays a resourceful master thief and Zeta-Jones plays an insurance investigator 
who sets an elaborate trap to capture him [Ebert, 1999]. A traditional script would result 
in Connery being caught and sent to jail. However, in the end Connery escapes capture 
by a strange twist of romantic intervention on the part of Zeta-Jones. Within the story 
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world of Entrapment, it is not difficult to imagine other exeiting stories with different 
eonelusions. 

2, Dramatic Beats 

Reeent work by Miehael Mateas of the Oz group and Andrew Stern has moved 
away from the plot graph paradigm for sequeneing story events [Mateas and Stem, 2000], 
[Mateas and Stem, 2002], They propose the design of an arehitecture for integrating plot 
and charaeters that is based on a dramatic beat. A beat is the smallest pieee of dramatic 
action that can occur. Scenes are eomposed of multiple beats. The exeeution of a beat 
eauses some “value” within the story environment to ehange, where a value is a property 
of an individual or relationship. Assoeiated with eaeh beat is a set of preeonditions 
neeessary for the beat to oeeur, a deseription of the values to be ehanged by the beat, 
eonditions used to determine sueeess or failure of the beat, and joint plans to be carried 
out by the characters in order to exeeute the beat. The joint plans are simply the 
behaviors neeessary for the eharaeters to earry out the beat. 

When the preeonditions have been met for a speeifie beat to oeeur and that beat is 
ehosen for exeeution, the drama manager aeeesses the “joint plans” for the speeifie beat 
and hands the appropriate eharaeters their behaviors neeessary to earry out the beat. The 
plans have been designed to aeeomplish the beat. This means that high-level goals and 
plans that drive an agent’s behavior do not reside within the agent, but are loeated in the 
beat. The drama manager pareels out the behaviors and goals to the agents partieipating 
in the beat. Low-level goals still reside within the agent (movement, personality moves, 
facial expressions, etc.). A set of related beats necessary to complete some larger aetion 
are grouped together in a scene. 

A seene eonsists of preconditions, a description of the values intended to be 
ehanged by the seene, a eolleetion of beats with whieh to eonstmet the seene, and a 
deseription of the dramatic arc that should be followed within the seene. Preeonditions 
test whether the seene is appropriate given the eurrent story and eharaeter state. 

Sequenees of seenes are ehosen by a drama manager. At any given time, the story 
is in a eertain state eonsisting of the eurrent story values and other global state 
information such as active eonversational topies, physieal loeations of eharaeters, etc. 


30 



The drama manager is continually aware of the story state and chooses the next scene by 
examining the list of unused scenes and selecting the one that satisfies the preconditions 
and whose value changes best match the global plot arc. 

3, Directed Improvisation 

Barbara Hayes-Roth describes improvisation as “a particular form of theater in 
which actors spontaneously and cooperatively generate their stories and their characters 
at performance time under the constraints of directions from sources such as the 
audience, predefined scenarios and other actors” [Hayes-Roth and Rousseau, 1997]. 
Stanford University’s Virtual Theater Project (VTP), under the direction of Barbara 
Hayes-Roth, has developed an architecture for building computer characters that perform 
directed improvisation. Synthetic agents are provided as intelligent actors that improvise 
their behaviors without detailed planning. The underlying agent architecture centers 
around a mind-body design. The mind is the implementation of a social-psychological 
model integrating personality traits, moods and attitudes affecting interpersonal 
relationships. The mind updates the agents’ knowledge based upon external inputs and 
stimuli, controls the agents decisions and provides input to the body. The body is the 
expression of the actions selected by the mind [Hayes-Roth and Rousseau, 1997], [Hayes- 
Roth eta/., 1995]. 

There are two types of improvisation characters: autonomous actors and avatars. 
The avatar provides the user with an interface into the story world. Both the actors and 
the avatar receive directions from a scenario and other actors. The autonomous actors 
decide their behavior based solely on their personality, mood, attitude, and the received 
directions. The avatar is primarily directed by the user who selects the actions to 
perform. However, the manner in which the avatar carries out the selected action is 
determined by the avatar’s personality, mood and attitude. 

A predefined story scenario is used to impose a story structure on the action. In 
an application of the Virtual Theater called CyberCafe, a restaurant scenario is applied 
with two autonomous actors and an avatar. One autonomous actor is a waiter, the other is 
a customer. The avatar plays another customer. The scenario provides the high-level 
direction to the actors, so as to cause major events to occur in a prescribed order. The 
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specific actions chosen by the autonomous actors to carry out the events are determined 
by the actor’s personality, mood, and attitude. Depending on the mood of the customer 
when the waiter brings his drink, the customer may do anything from throwing the drink 
back at the waiter, to thanking him and providing a tip. The variability in the selected 
actions manifests itself as improvisation. 

The work of the VTP deals with a specific type of drama, focusing heavily on 
character and less on variability in the plot. In fact, the plot is defined in advance and 
remains static throughout the story. 

4. Verb-Centric Interactive Story-Telling 

In the Virtual Theater Project, Barbara Hayes-Roth capitalizes on the unique 
personality and motivations encoded into each agent to tell a story from a fixed script. 
The characters’ mannerisms and personalities are driven by their interactions with each 
other and the user’s avatar. The stories are interactive because the agents’ moods and 
reactions are affected by the environment and the other characters. The character 
interactions take on a story form through the imposed script. 

While still relying heavily on believable agents that react to their environment, the 
Oz project takes story plot a step further, allowing the plot to change based on previous 
events and the prognosis for the remaining events forming a good story. Chris Crawford 
has designed a storytelling system, Erasmatron that takes a wholly unique approach to 
interactive storytelling [Crawford, 1999]. 

Crawford’s approach is founded in the assumption that storytelling is inextricably 
the function of an artist, for which no algorithm can be designed to truly replace the 
human story teller. Storytelling relies heavily on contextual knowledge of the culture in 
which it is told. Crawford states “Just as the meaning of language is steeped in culture, 
so too must the story teller integrate the story in the audience’s culture. For this reason, 
interactive storytelling with all of its creative responsibility must always be the domain of 
the artist and not an algorithm” [Crawford, 1999]. 

Erasmatron, as described by Crawford, is a verb and sentence based system. The 
story is organized around a master list of verbs for the story world. For each verb, the 
author declares a set of character roles necessary to carry out the verb action. For 
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instance, a sentence with an action verb elause of “assault” might have eharacter roles of 
attaeker, vietim and bystander. In this system, artistie functions are segregated from 
algorithmie functions. The artist sets dramatie goals (artistic function) that require the 
eharaeters to take action, while an algorithm handles the physieal implementation or 
realization of the aetion. Continuing with the verb elause “assault,” for the attaeker to 
aeeomplish their goal they will need to move to the vieinity of the vietim. The artist 
initiated the eharaeter’s need to move, and an algorithm eontrols the charaeter’s 
movement. 

Dramatic goals are established through inclination functions coded by the author. 
These funetions form the basis of the eharaeter’s deeision proeess. When faeed with a 
choiee, the autonomous eharaeters referenee an inclination function to resolve the 
decision. Erasmatron provides the environment for designing the story world, but the 
definition and eoding of the inclination functions is left to the artist. 

While the system allows for eomplex behavior and artistie influenee, Crawford 
aeknowledges “I have been hoisted by my own petard in the matter of eomplexity; as yet, 
largely beeause of its eomplexity, not a single artist has eompleted ereation of a viable 
story world using the Erasmatron!” [Crawford, 1999]. As a result, he has moved to an 
intermediate stage he ealls “assisted storytelling.” The latest version of Erasmatron has 
stepped baek from requiring the artist to eode inelination algorithms to govern the 
ehoiees for the eharaeters. The artist’s role is now redueed to defining dramatieally 
viable options for the eharaeters, while the user makes the actual choices for the 
eharaeters. 

5, Interactive Stories with Strong Pedagogical Goals 

As previously discussed, interaetivity and plot ean be balaneed in a number of 
ways. In Hayes-Roth’s VTP, interaetivity is expressed through the diversity of agent 
responses to a given situation. Interaetivity oecurs at the aetion seleetion level, while the 
basie plot remains unehanged. In the Oz projeet, interaetivity manifests itself at the plot 
level. By defining a set of plot events and eomputer events that will tell the story, 
interactive input by the user impaets the sequeneing of plot events in real time. The 
seript ehanges, but only in the ordering of events. A dramatieally appealing story 
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emerges as a result of the drama manager’s seleetion proeess of events. In this seetion, 
two approaehes are examined for presenting stories with strong pedagogieal goals. In 
both oases, aohieving the presoribed learning objeotives is of prime importanoe, as a 
result the interaotive freedom of the partioipant is oonstrained to maintain the integrity of 
the soript. 

a. A Story Deconstruction Approach to Interactive Drama 

A group of researohers from the Center for Advanoed Researoh in 
Teohnology for Eduoation (CARTE) at the University of Southern California’s 
Information Soienoes Institute have been working on an agent-based approaoh to oreating 
interaotive pedagogieal drama [Marsella et al, 2000]. In their system, interaotivity is 
provided on two levels, the oharaoter level and the story event level, while the narrative 
struoture of the story is maintained through a soript deoonstruotion soheme. Story 
oharaoters are free to ohoose their aotions autonomously, while direotor and 
oinematographer agents manage the aotion and its presentation in order to maintain story 
struoture, aohieve pedagogieal goals and aohieve the best dramatio effeot. There are four 
main oomponents to the system: autonomous oharaoter agents, the user or person 
learning, a direotor agent and a oinematographer agent. These oomponents have been 
brought together in a multimedia title oalled Carmen’s Bright IDEAS. 

The story is provided in a “presentational” style where the user oontrols 
the intentions of one of the oharaoters, but does not partioipate direotly in the first person 
as a oharaoter. This allows a level of interaotivity speoifio to the user’s oharaoters. 

Event level interaotivity is introduoed by starting with a full story soript 
and suooessively deoomposing it into smaller and smaller pieoes, all the while identifying 
plaoes where variability oan be introduoed while remaining true to the pedagogieal goals 
of the soript. This allows interaotivity in the plot while maintaining the story struoture. 
Eundamental to the deoomposition proeess is determining whioh variations are desirable, 
either from the pedagogieal perspeotive or from the dramatio perspeotive. This helps in 
defining the aotions of the direotor agent to avoid undesirable variations. 

The soript is a sequenoe of aots, where eaoh aot is a sequenoe of soenes. 

Eor a soene to be realized, oertain events must ooour within the soene. The events 

themselves are motivated by the goals of the individual oharaoters partioipating in the 
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scene. The seript is broken down into a hierarchieal narrative structure, starting with the 
acts and continuing all the way down to analyzing the goals that can cause the events to 
occur that make up a scene. Onee deeomposed, the designers determine where variability 
can be introduced. For example, alternative scenes or a different ordering of scenes may 
be able to achieve the pedagogical goal of the act. Similarly, different patterns of events 
can achieve the same scene goal. 

It is the job of the director agent to select the specific actions for the 
agents to present a dramatically appealing story. The director in essence controls a 
discrete event driven simulation, where the discrete events are individual character dialog 
turns. The director has global knowledge and selects the next event based upon the 
internal state of each character and state of the environment. With this state information, 
the director selects the actions for the characters based upon the previous script 
decomposition. 

A cinematographer agent manages the presentation of events to maximize 
their dramatic impact. The agent gets a filming deseription from the character agents at 
each dialog turn. The filming description sets the action that needs to take place in the 
next turn. The description contains information such as who is speaking, what is said 
(and how long it will take), non-verbal gestures that will occur, any dialog boxes 
(eharacters thought boxes) and flashbacks that must be shown. Based on this, the agent 
references a set of predefined rules to determine what to show, how to show it and when. 

By starting with a well-defined script, the story remains true to the 
pedagogical goals while supporting interactivity at the plot level. In essence, the 
decomposition process yields different means to the same end. 

b. Predefined Networks with Confined Freeplay 

The Institute for Creative Technologies at the University of Southern 

California is a U.S. Army funded institute with a mandate to bring together the resources 

and talents of the entertainment and game development industries with computer 

scientists in order to advance the state of immersive training simulation [ICT, 2002]. A 

keynote project of ICT is the Mission Rehearsal Exercise (MRE) system, a virtual reality 

training environment that combines virtual humans [Rickel et al., 2001], wide screen 

graphics and immersive audio to present a real-world training scenario. Underlying the 
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MRE system is a network-based interaetive story system called StoryNet [Swartout et al, 

2001 ], 

As previously discussed, interactive narrative carries the inherent problem 
of balancing narrative control with interactivity. In the case of military training systems 
such as the MRE system, this challenge is exacerbated by the need to ensure doctrinally 
correct training objectives are met. These training requirements place an additional 
restriction on the level of interaction available to the participant. ICT’s approach is 
similar to that used in CARTE’s Carmen’s Bright IDEAS. The MRE system starts with a 
well-structured script that meets the prescribed pedagogical goals. The script is then 
structured into a network of interactive “freeplay” nodes and linear transition links that 
are used to setup the follow-on “freeplay” node. Associated with each transition link are 
gating conditions used to determine when the participant has met the conditions 
necessary to transition to another node (Figure 7). 

In this system, the nodes correspond to tasks the participant in intended to 
learn, or key decision points in the scenario. Within the nodes, the participant is free to 
interact with the virtual humans and make decisions. Extending from each node are 
linear movie links. Once the interaction is complete and the participant has met the 
gating conditions (either through a specific decision or completion of a task), they take on 
a passive role while being transported to the next node via one of the linear movie links. 
By limiting the options available in the freeplay nodes and the number of links extending 
from each node, the designers are able to control the scope of behaviors the system must 
handle. 
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link enable transition 

Figure 7, StoryNet Architecture from [Swartout et al, 2001] 

The MRE system is an extremely effeetive training environment. The 
8.75 by 31 ft. curved screen, coupled with intelligent characters, real-time animation and 
64-track spatial sound creates an almost holodeck-like environment. However, this level 
of immersion comes at a price in terms of story complexity and plot variation. Given the 
extensive scripting required to meet the training goals and develop movie links, 
expanding the system appears to be both time consuming and expensive. Adding a new 
decision option to an existing node requires a script to be written and a linear movie 
developed for the associated outgoing link. Similarly, if an entirely new node is added to 
the network (new decision or training objective), a host of links and supporting nodes 
must be added. 

F. AGENT-BASED PLANNING SYSTEMS 

Generally, multi-agent systems can be divided into two kinds of planning 
schemes: static and dynamic. Planning is considered dynamic if the plan evolves during 
execution of the simulation model. Static plans are pre-arranged, i.e., defined before 
simulation runs take place. 
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A hybrid approach combines static and dynamic planning as proposed by a 
number of others, most reeently in MEAGENT [Rowe et al., 2000], [Andrade, 2000]. 
ME AGENT is a Prolog system using means-ends analysis. MEAGENT agents are given 
tree-struetured plans by their designers, but then allowed to adapt. Agents are given plans 
that are designed to seek goals by dividing the goal into sub-goals, and sub-goals into 
smaller sub-goals, ete. The root of the goal tree represents aehievement of the top goal, 
and the leaves of eaeh tree represent one step in aehieving a sub-goal. When appropriate, 
the tree is re-struetured, dynamieally, in a proeess that Rowe and Andrade eall “re¬ 
planning.” Re-planning in MEAGENT was used in training sailors how to fight fires 
onboard ships. 

MEAGENT adapts to unexpeeted events during the simulation, but it does not 
invent plans. Eaeh simulation run begins with a statie pre-defined plan (goal tree), whieh 
must be designed by the human operator. While means-ends analysis provides flexibility 
and adaptability, it doesn’t solve the problem of plan diseovery. 

Another approaeh that is elosely related to this work is PEANGENT [Oshuga et 
al, 1997]. Mobile agents in PEANGENT have planning eapabilities that are a by-produet 
of adapting to new web-based environments. The authors elaim that this kind of 
adaptability and re-planning make agents “intelligent.” 

Aeeording to [Oshuga et al, 1997], 

PEANGENT has an advaneed planning feature - speeifieally, a teehnique 
that uses baekward reasoning from deelarative statements of user 
requirements to generate sequenees of aetions that will satisfy the 
requirements. The planning meehanism is refieetive, that is, agents ean 
exeeute metalevel planning, or meta-planning. 

PEANGENT allows ineomplete plans, thus, aetions ean be delayed until 
additional information is gathered by the mobile agent, and the aetion resolved during 
exeeution. PLANGENT agents ean make a least commitment plan that delays definition 
of whieh aetion to perform until the agent has visited enough web sites to deeide. Still, 
plans are limited by top-level requirements as defined by the user. PLANGENT’s 
dynamie planning steps are deseribed in Eigure 8. 
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1. Express user requirements in terms of constants, 
variables, and predicates. 

2. Initialize a plan as a set of actions and constraints. 

3. The agent tries to satisfy an un-realized goal by 
choosing an action from a plan or by choosing an action 
that is not in its original plan. 

4. If an action was selected in step 3 that is not in an 
original plan, that action is considered a threat if it 
breaks the consistency of the original plan. A threat 
action is handled by gathering more information or 
backtracking. If no more actions are available, the 
agent backtracks to step 3. 

5. The current plan is executed. 

6. Generate a metaplan: If pre or post conditions are not 
met, declare failure. If infinite loops are found, 
terminate the planning process. If failure, re-planning 
is required. 

Figure 8, PLANGENT Planning Steps from [Oshuga et aL, 1997] 

PLANGENT has been applied to airline reservation systems where goals are 
destinations, travel dates and times, and eonstraints are eost, etc. Plans are represented as 
trees with sub-goals derived from top-level user input requirements. 

G. SUMMARY 

In this chapter, work related to the design and implementation of interactive story 
systems was presented. An overview of interactive entertainment and narrative 
intelligence was provided along with a description of plan-based story-telling and story¬ 
understanding systems. While this work served as a forerunner to interactive stories, 
many other fields of study have played key roles or provided motivation behind research 
into developing interactive story systems. Some of these include human-computer 
interface design [Laurel, 1991], knowledge understanding and representation [Schank 
and Abelson, 1977], AI planning [Young, Pollack, & Moore, 1994], narrative theory 
[Aristotle, 330 BC] [Freytag, 1900] [Polti, 1977] and interactive cinema [Brooks, 1999] 
[Davenport et al., 2000]. The range of influence is too great to present a thorough 
treatment of each of these in this dissertation. 

The interactive story systems presented in this section were described with an 
emphasis on the approach used to balance interactivity with narrative control to present a 
story. This examination reveals that existing systems either don’t scale to large 
complicated stories (i.e., only a small number of stories are possible), or they are based 
on a fixed plot. A scalable architecture supporting variable story lines that can be 
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adapted to multiple simulation domains does not yet exist. The goal of this researeh is to 
develop a story world where literally thousand of stories are possible. To aeeomplish 
this, the top-down structure employed by previous systems has been abandoned in favor 
of a bottom-up design paradigm based on multi-agent system simulation techniques. 

While a review of the previous work revealed areas requiring attention, it also 
served to provide inspiration, both in terms of what has worked in the past, and in what 
has not. The common theme of a central drama manager shared by virtually every 
project described, and the inherent problems with scalability, led to the exploration of a 
bottom-up, distributed control architecture for generating stories. A second common 
thread running through a majority of the projects was the notion of a fantasy world, or a 
coherent space in which the story (or scene) takes place [Laurel, 1986], [Weyhrauch, 

1997] , [Mateas, 1997] and [Hayes-Roth and Rousseau, 1997]. The concept of a story 
world evolved into an environment for a multi-agent system. Finally, a study of related 
work, as well as additional readings in narrative theory and interactive narrative [Murray, 

1998] , [Polti, 1977] revealed an obvious, but fundamental relationship between an actor, 
character, role and scene. An actor takes on the personality of a character, while the 
actions available to the character are a function of, and bounded by, the scene and the role 
the character plays in the scene. This actor, character, role, scene relationship, as it 
applies to the story engine, is described in Chapter VI. 

The remaining chapters present a scalable multi-agent system architecture that 
balances interactivity, with character autonomy and story world controls, to present the 
user with a personalized story. 
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III. MULTI-AGENT SYSTEMS 


A, INTRODUCTION 

This chapter presents a definition of multi-agent systems (MAS) that serves as a 
basis for defining the MAS architecture used by the story engine to generate interactive 
stories. 

MAS are composed of numerous interacting computing elements, known as 
agents. Agents are eomputer systems with two important eapabilities. First, they are, at 
least to some extent, eapable of autonomous aetion - of deeiding for themselves what 
they need to do in order to satisfy their design objeetives. Seeond, they are eapable of 
interacting with other agents - not simply by exehanging data, but by engaging in 
analogues of human soeial activity: cooperation, eoordination, negotiation, and the like 
[Wooldridge, 2002]. 

MAS operate from the bottom-up, using multiple adaptive agents “...(as) 
intelligent aetors, interacting among themselves by using their defined attributes and 
methods, but (are) able to modify those eonstraints to meet the goals assigned them by 
the modeler...providing real insight into how best to encourage and take advantage of 
individual initiatives and adaptability “ [Weiss, 1999]. 

MASs have no eentralized eontrol - the agent simulation is leaderless. Eaeh 
agent in the simulation independently pursues its own independent goals. Some agents 
may cooperate, while others eompete. The result is a highly dynamie environment where 
software agents, with no human intervention, can explore an environment, interacting 
with other agents and object in the environment in pursuit of their goals. The outeome is 
innovative solutions for achieving the goals [Hiles et al., 2001]. 

While there are many definitions of multi-agent systems with varying degrees of 
formality, Ferber’s definition of a MAS (given in the next seetion) is used as a basis 
throughout this dissertation [Ferber, 1999]. Additional general information on multi¬ 
agent systems can be found in [Weiss, 1999], [Holland, 1995] and [Russell and Norvig, 
1995]. Deseriptions of speeific MAS architeetures and simulations ean be found in 
Swarm [Langton, 1997], ISAAC eombat simulation [Ilaehinski, 1997], and Echo 
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simulated world [Echo, 2000]. These examples primarily explore the use of MAS 
architectures to model emergent behavior. 

B, MULTI-AGENT SYSTEM DEFINITION 

Ferber describes a multi-agent system as “an electronic or computing model made 
up of artificial entities which communicate with each other and act in an environment.” 
More precisely, multi-agent systems can be defined as a set of interacting elements 
described by Equation 1. 

MAS = (E, O, A, R, Op, Laws} 

E - Environment 

O - Objects situated in the environment 
A - Agents, (A c O) 

R - Relations linking objects O 
Op - Operations 

Laws - Constraints governing the environment 

Equation 1. Multi-Agent System from [Ferber, 1999] 

1, Environment 

MAS simulations can be formulated in a situated or non-situated environment. In 
a situated simulation, the environment may be a Euclidean n-space or a notional space 
appropriate to the given simulation. Regardless, at any given time, the agents and objects 
have a “location” in the environment. This may be an (x, y, z) coordinate in the case of a 
3-D environment, or it may be a more abstract “location” defined in terms of time and 
resources. In a situated simulation, agents are capable of perceiving their environment, 
recognizing the objects and agents populating the environment, and transforming the state 
of the environment by interacting with the other agents and objects. 

In the case of non-situated MASs, the environment is populated with agents and 
objects that interact in accordance with defined relations through agent-to-agent and 
agent-to-object communications. The objects are typically resources needed by the 
agents to achieve their goals. The agents and objects need not have a “location.” The 
MAS is defined by the behavior of the agents and the network of relations linking them 
together. Ferber describes this type of MAS as a “purely communicating MAS.” 
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2, Objects 

Objects are described as passive entities situated in the environment that can be 
perceived, created, destroyed and modified by the agents. Under Ferber’s definition, 
agents are a subset of the objects with no clear distinction between the two. Objects can 
be thought of as non-agent entities in the environment. They may be totally passive like 
a rock or a tree, or they may be active as is the case with a radio transmitter emitting 
signals into the environment. The primary distinction between objects and agents is 
intent. Objects may be able to act to modify the environment, but there is no autonomous 
intent behind the actions. 

3, Agents 

Ferber describes an agent as a physical or virtual entity that can act in an 
environment, communicate with other agents, is driven by internal goals and objectives, 
possesses resources, perceives its environment but only a partial view (or possibly none 
at all), has skills, may reproduce, and behaves in a manner that satisfies its objectives. 
His definition makes no distinction between cognitive and reactive agents, and is general 
enough to encompass both. 

Cognitive agents (or deliberative agents), from the distributed artificial 
intelligence (DAI) community, are traditionally based on first-order predicate logic, 
sophisticated reasoning, and rely on the internal manipulation of symbols. These agents 
maintain a symbolic representation of the environment within which they operate, and 
focus on communication and cooperation between agents. Most importantly, these 
agents have intentions - goals and plans to achieve goals [Hiles et al., 2001]. 

Reactive agents, from the field of artificial life (A-Life), are reflexive - actions 
are “reactions” to stimulus regulated by perceptions and the agent’s internal state. These 
agents maintain no planning, history, or symbolic representation of the world. The 
simple reactive agents are combined into a society, where intelligence is seen as emergent 
from the vast interactions of the agents and the environment. See [Weiss, 1999] for a 
more detailed comparison of cognitive and reactive agents. 

Throughout the remainder of this dissertation, both objects and agents are 
referenced extensively. Often it is convenient to make statements that apply to both 
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objects and agents. When both are being referred to together, the term entity will be 
used. From a software engineering perspective, an object is an entity and an agent is an 
entity. 

4. Relations 

Relations are abstract links that create a dependency between the agents. They 
can be as simple as allowing communications between two agents or they can be 
complicated relationships establishing the rules of cooperation and competition among a 
society of agents. For example, two agents can form a cooperative relationship to pool 
resources that will allow a common goal to be achieved that would not be possible by any 
one agent acting alone. In the case of a simulation involving a military command 
structure, the chain of command is a complicated relationship that imparts certain 
responsibilities on the agents based on their position in the structure. An agent filling a 
“command” role will inherit certain goals that go along with the command position. In 
addition, the command relation carries with it a certain amount of influence over the 
agents in subordinate positions; the goals of the “commander” agent influence the goals 
of the “subordinate” agents. A detailed discussion of relations as they apply to multi¬ 
agent systems can be found in [Roddy and Dickson, 2000]. 

5. Operations 

Agents are capable of autonomous actions or “operations” that allow them to 
perceive their environment, interact with each other and objects, and change their state, 
either internally (mood swings from happy to sad) or relative to the environment (move 
from location x to location y). The set of actions available to an agent is dynamic. It is 
constantly changing based on the current context, where current context is a function of 
the agent’s state and that of the environment. That which is appropriate at time t, may 
not be appropriate at time t+1. The change to the set of appropriate actions could be 
because the agent’s state changed, or it may be the result of a change in the environment. 

Given a MAS simulation with environment E, if Qa is the comprehensive set of 
possible actions for agent a, then Equation 2 describes the reduced set of possible actions 
a can take at time t. This set of actions, Ela,t, is a function of agent a’s state at time t, 
^(at), and also the state of the environment that is within the given agent’s scope of 

44 



knowledge E] is the agent’s view or perception of the environment at time t. 

Included in '^{E'i) is the state of all of the agents and objects in ^'as perceived by a. As 
stated before, agents only have local perspective; they do not normally have complete 
knowledge of their environment so E. 

na,t =M(at), ^(E't)); a,^na 

Da,t = the set of valid actions for agent a at time t 
f2a = set of all possible actions for agent a 
^(at) = the state of agent a at time t 
^(E't) = the state of environment E' at time t 

E 'is agent a’s perceived environment (E'^E) 

Equation 2. Action Set for Agent Type a at Time t 

While the set of all possible actions for an agent may be very large (f3a), the 
contextually appropriate actions at any given time {Oaf should be a much smaller set. It 
is from this subset {Oa,t c Oa) that the agent decides what to do based on its active goals. 

6, Laws 

Given the operations described above, the rules for applying the operations, along 
with the reaction of the system to the operations, are captured in the laws of the system. 
These laws are the limitations and restrictions the agents and objects must adhere to while 
they reside in the environment. They might include issues related to physically based 
modeling such as collision detection, gravity and light propagation, or they may govern 
the way relations are created and destroyed [Roddy and Dickson, 2000]. 

C. SUMMARY 

This chapter presented a general framework for describing multi-agent systems 
based on Ferber’s definition. MASs were described in terms of their environment, the 
entities populating the environment (agents and objects), the relationships established 
between the entities, and finally the actions agents can take (operations) and the rules by 
which the actions are applied (laws). In the next chapter, the design philosophy 
underlying MAS research at the MOVES^ Institute is presented along with specific 
design concepts that are key to this research. 

2 MOVES (Modeling of Virtual Environments and Simulation) Institute, Naval Postgraduate Sehool, 
Monterey, CA. 
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IV. MULTI-AGENT SYSTEM RESEARCH 


A, INTRODUCTION 

For the past three years, the MOVES Institute’s Computer-Generated Autonomy 
Group has been exploring MAS arehitectures that faeilitate the development and eontrol 
of complex, adaptive behavior. This chapter introduces four agent-based simulation 
design concepts, composite agents, goal management, tickets and connectors, for 
modeling multi-agent systems and implementing the models in software simulations 
[Hiles et al., 2001]. These concepts have evolved over the past three years into a novel 
simulation methodology called Connector-based Multi-Agent System (CMAS) 
simulation that is capable of generating dynamic plans and interactive stories. The 
CMAS architecture, described in Chapter V, serves as the underlying model for the story 
engine. 

Portions of the following text have appeared previously in two collaborative 
papers; [Hiles et al., 2001] and [VanPutte et al, 2001]. 

B. SEMI-FLUID SOFTWARE STRUCTURE 

Interactive stories, along with their underlying plots, are ever changing and fluid 
structures that only exist in the past. That is, at any given instant in time, it is possible to 
precisely describe what has already occurred, but not will happen next. In essence, 
stories are generated one step at a time based on the complex interactions of the 
characters with each other and their surroundings. The structure of an interactive story is 
not fixed until after the story is told. 

Software development has traditionally focused on building software that is in 
direct conflict with the notion of fluid structures. Software systems are typically 
engineered based on rigid designs where fixed and immutable relationships are 
established among the components inside the software. It is assumed that the structure 
must be tightly bound at design time if a program has any chance of meeting its design 
goals. This outlook can be described as analogous to a new highway system that is 
designed on paper and constructed with concrete and steel to meet the forecast needs of a 
growing city. Once built, the highway system remains fixed and static unless new 
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construction occurs. It would be absurd to expect it to mold itself into new forms to meet 
growing infrastrueture and ehanging traffie patterns. 

This same thinking underlies mueh of the work in developing plans and produeing 
interaetive stories. The arehiteeture of the plot is fixed at design time; its strueture is 
basieally inert. While varying degrees of deviation are possible based on the systems’ 
design, in general, the only stories that ean be generated are those that the author or 
designer has thought of a priori. The systems are not eapable of produeing stories that 
were not laid out in advanee. 

With multi-agent systems, it is possible to build software that modifies its own 
strueture, within a set of eonstraints, to maintain close eontaet with a dynamie 
environment. In the ease of interaetive stories, this entails a MAS arehiteeture that allows 
the story system to generate a fluid plot that is sensitive to the dynamie relationships 
between the eharaeters, and the eharaeters with their environment, while remaining true 
to the basie eonstraints (or laws) of the story world. 

1. Indirect Solutions - A Design Paradigm Shift 

Most software developers and programmers have been trained in traditional 
software engineering, relying on struetured system designs that implement a direct 
solution to the problem. Traditional problem solving in software engineering is direct in 
the sense that the developer eoneeives of an algorithmie solution and transfers that 
solution to software. Software development rigor and praetiee is used to ensure the eode 
will produee an exaet exeeution of the algorithm. In direet solutions, the programmer 
knows exaetly how to solve the problem and the software implements that solution 
preeisely. This approaeh is fine for problems where the domain is well known, and the 
relationships are statie, finite and well defined. Direet solution systems are somewhat 
eomparable to well-behaved funetions; for a given input, the designer knows what to 
expeet for the output. Surprises are a elear indieation of a bug in the system. 

In sharp eontrast, surprises in MAS simulations are not only aeeeptable, but are 
the desired end, as long as the system operates within boundaries that are explieitly 
determined. The software is intended to surprise the designer within a system of 
eonstraints. This is possible through the use of software agents that diseover an indirect 
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path to the solution, thereby allowing for the possibility of arriving at a solution the 
designer may not have previously considered. In this way, multi-agent systems are 
capable of producing innovative solutions. These solutions are indirect in that they were 
not explicitly programmed into the software; rather they are solutions that are consistent 
within the constraints the designer places on the software agents. As a result, any 
solution that is valid within the imposed constraints, is no longer a bug, but a potential 
novel approach to the problem. This is precisely the design philosophy that inspired the 
story engine. A story world is defined and novel story lines {surprises) are generated as a 
result of agents pursuing their goals {indirect solutions). 

C. COMPOSITE AGENTS 

Multi-agent system simulations typically consist of numerous high-level agents 
that represent entities operating in a common, shared environment. The agents residing 
in this “outer environment” interact with one another and the objects in the environment. 
They sense their environment, interpret the sensory input and decide what actions to take. 
These actions, in turn, affect the environment either directly through agent-to- 
environment interactions or indirectly through agent-to-agent interaction. In an effort to 
capture the strengths of both cognitive and reactive agents, while at the same time 
simplifying the design of such a complex agent, the Composite Agent (CA) architecture 
was developed. 

CAs are composed of a combination of agents (Figure 9). They contain a set of 

Symbolic Constructor Agents (SCAs) that work with sensory streams (or impressions) 

from the outer environment to create a symbolic inner environment (Emner) representing 

the agent’s perspective of the outer environment (Eouter). The basic structure of a 

Composite Agent follows Wooldridge’s “observe - update state - acf’ model for agents 

that maintain state [Wooldridge, 2002]. The SCAs define the agent’s sensor capabilities 

and are tailored to sense specific aspects of the environment. They also act to control and 

filter impressions of the outer environment, so the agent isn’t overwhelmed in a rich outer 

environment. Emner is influenced not only by what the SCAs sense, but also by the CA’s 

internal state. Eor instance, in a predator-prey simulation, if the predator is hungry and 

senses an animal, it would show up in Einner as food. On the other hand, if the predator 

has just eaten, then the animal would appear as just another animal in Einner- 
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Figure 9, Composite Agent 

The symbolic inner environment is the agent’s perception of the shared outer 
environment within which it operates. Einner has little resemblance to the actual outer 
environment, rather it is an encoding of Eouter optimized to suit the CA’s specific 
function. The role of an SCA is not unlike the role of radio navigation aid used by a 
pilot. The navigation aid senses radio signals in the outer environment and converts them 
into directional information that the pilot can use to navigate the aircraft. The inner 
environment used by the pilot for making decisions has little resemblance to the view 
looking out the window, but it is optimized for use by the pilot in navigating the aircraft. 

Combined with the SCAs is a set of Reactive Agents that operate on the symbolic 
inner environment and generate actions for the CA to perform. Each Reactive Agent has 
a set of possible goals and an apparatus for managing the process of selecting the active 
goal or goals. 

D, REACTIVE AGENTS AND GOAL MANAGEMENT 

Composite Agents contain a set of Reactive Agents (RAs), where each reactive 
agent is responsible for promoting a specific behavior of the Composite Agent. The set 
of RAs taken as a group, define the CA’s set of high-level behaviors. The RAs operate 
within the world of the inner environment. They take as input sensory information from 
E inn er, and producc as output actions for the agent to perform. 
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Each RA has one or more goals speeifie to furthering its behavior or funetion. So 
at any given time there are numerous goals eompeting for the CA’s attention. Just as 
humans have multiple goals (sometimes eonflieting), an agent too ean have multiple 
goals it wishes to satisfy. In human deeision-making, goals are eonstantly shifting in 
priority, based on the person’s eontext and state. Agents ean mimie the flexibility and 
substitution skills of human deeision-making with a variable goal management apparatus 
within the RAs. Thus, eontextually appropriate, intelligent behavior emerges from this 
goal apparatus. RAs interpret the symbolie inner environment and through their goal 
apparatus, proeess this information to balanee their goals and return an appropriate aetion 
for attaining their highest priority goal(s) (Figure 10). 

The basie definition of a goal has four eomponents: a state, a measurement 
method, a weight, and a set of aetions for aehieving the goal (Equation 3). 

goal = < s, mm, w, {a}> 

s = state e {inactive, active, achieved,...} 
mm = measurement method 
w = weight 

{a} =an action set for achieving the goal 

Equation 3. Goal Definition 

The goal’s state is an indieation of whether the goal is aetive, inaetive, aehieved, 
or in some other domain speeifie status. The measurement method translates the sensory 
input reeeived by the RA into a quantifiable measure of the eurrent strength of the goal 
and how well it is being satisfied. This permits an agent to prioritize goals and adjust 
goal states based on eontext. A goal may also have a weight attaehed that ean be used to 
adjust the importanee or priority of the goal based on experienee. Tied to eaeh goal is an 
action or set of aetions for aehieving the goal under varying eireumstanees. The end 
result is that within the RA goal apparatus there are multiple goals that are eonstantly 
ehanging - moving up and down - with the top (aetive) goals dominating the agent’s 
behavior. 
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Figure 10, Reactive Agent 

Additionally, agents can discard behaviors that do not further their goals, and 
increase the use of behaviors that have proved successful in reaching goals. This simple 
process serves as a reactive learning system where the agent learns from the environment, 
based on “what works,” with no human expertise or intervention. 

Goal switching based on a dynamically changing environment produces 
innovative and adaptive behavior. The ability to continually adapt to an ever-changing 
environment in real time is provided through a construct called connectors. 

E. CONNECTORS 

This section presents a formal definition of a connector and describes its 
fundamental behaviors. [VanPutte, 2002] extended the definition of a connector to 
develop iconnectors that are used as an inter-entity communications mechanism in the 
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information assurance domain. In the course of his work, VanPutte presents a Unified 
Modeling Language (UML) representation of iconnectors and their funetion. In addition, 
he describes a graphieal notation for visualizing iconnectors that is used here to visualize 
connectors. 

1. The Biological Inspiration Behind Connectors 

Multi-agent simulations are used to model inherently eomplex systems. One of 
the major ehallenges is developing a eommunieation protocol capable of standing up to 
the potentially eombinatorial explosion in sensory information that floods the 
environment as the agents internet with eaeh other and their surroundings. The protoeol 
must be flexible enough to allow agents to define what information they need and when 
they need it. This is partieularly diffieult in agent-based modeling where the entities are 
autonomous and eapable of changing their seope of interest to suit their current goals. 

The complexity described above and assoeiated eommunieation requirements are 
not unlike that found in molecular biology when studying eell-to-eell eommunieation, 
eoordination and eontrol. The following exeerpt was taken from a moleeular biology text 
and eaptures the fundamental role signaling plays in eellular biology. 

...the behavior of eaeh individual eell in multieellular plants and animals 
must be earefully regulated to meet the needs of the organism as a whole. 

This is aeeomplished by a variety of signaling moleeules that are seereted 
or expressed on the surface of one cell and bind to receptors expressed by 
other eells, thereby integrating and eoordinating the funetions of the many 
individual eells that make up organisms as eomplex as human beings. 
[Cooper, 1997] 

The signaling meehanisms found in moleeular biology are the foundation of a 
remarkable intraeellular and intercellular eoordination and eontrol system that manifests 
itself in the form of a funetioning human body. This system serves as the inspiration 
behind a software agent eommunieation meehanism ealled connectors. The remainder of 
this seetion presents an overly simplified, and primarily pietorial introduetion into the 
signaling meehanisms used for eellular eommunieation. 

All eells reeeive and respond to signals from their surroundings. They respond to 
signaling moleeules seereted by other eells, allowing cell-to-cell eommunieation. Some 
of the molecules earry signals over long distances, while others act locally to convey 
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information between neighboring cells. There are four major types of signaling. 
Signaling over long distances occurs through Endocrine signaling, where molecules are 
secreted by cells and carried through the circulatory system to target cells a great distance 
away (Figure 11 A). In Paracrine signaling, the released molecules act on neighboring 
target cells (Figure IIB). Contact-dependent or direct cell-to-cell signaling occurs when 
the signaling molecule remains attached to the signaling cell (Figure IIC). Finally, 
Autocrine signaling takes place when a cell produces a signaling molecule to which it 
also responds (Figure IID). 


(A) Endocrine signaling 



(B) Paracrine signaling 





Figure 11. Cell Signaling Methods from [Cooper, 1997] 

Cells in multicellular organisms are typically exposed to hundreds of different 
signals in an environment that can act in millions of combinations. The cell must respond 
to this plethora of signals selectively, according to its own specific character (Figure 12). 
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Figure 12, Cells Responding to Combinations of Extracellular Signals from 

[Alberts et al., 2002] 

A cell may be programmed to respond to one combination of signals by 
differentiating, to another eombination by multiplying, and yet another by performing 
some speeialized funetion. The hundreds of signal moleeules that animals make ean be 
used to ereate an almost unlimited number of signaling eombinations. The use of these 
eombinations to eontrol eell behavior enables an animal to eontrol its eells in highly 
speeifie ways by using a limited diversity of signal moleeules [Alberts et al., 2002], In 
other words, from a limited set of signals, it is possible to eontrol extremely sophistieated 
and speeialized behavior. 

One way this is aeeomplished is through signal easeading. Figure 13 shows how 
an extraeellular signal moleeule ean trigger a easeade of intraeellular signals. The 
extraeellular signal moleeule at the top of the diagram binds with a reeeptor protein 
extended from the eell membrane. This single external interaetion begins a signal- 
response process that results in gene transeription oeeurring within the nueleus of the eell. 
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extracellular signal molecule 



Figure 13. Intracellular Signal Cascading from [Alberts et al., 2002] 

This section provided a brief description of cellular signaling to demonstrate how 
a system of signals and receptors can be used to control complex behavior. The next 
section describes a software construct called a connector that is used to mimic certain 
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aspects of cellular signaling. Connectors are not intended to duplicate exactly each of the 
cell signaling methods; rather they are an abstraction that captures the general nature of 
the distributed communication and coordination protocol. 

2, Connector Definition 

Connectors are a software component that can be “bolted on” to agents and 
objects in a multi-agent simulation to provide a communication and coordination 
capability. The base set of connector behaviors can be augmented for the specific 
simulation to provide a powerful and flexible domain specific communications protocol. 
Through a process called connecting, agent-to-agent and agent-to-object connections are 
established during which two-way communication occurs between the connected entities. 
Connectors are active objects that sense and react to the environment. They activate 
(extend) and deactivate (retract) in response to changes in the state of the entity to which 
they are attached. As the entity’s state and the state of the environment changes, the 
connectors sense the changes and extend or retract accordingly. 

Equation 4 defines a connector in terms of seven components. Connectors are 
attached to a host entity and have an associated control function that defines its behavior. 
Connectors, at a minimum, implement four basic actions: extend, retract, connect and 
disconnect. These actions are used to modulate the connector between its three primary 
states of extended, retracted and connected. Connectors are defined by type and can 
extend in one of two modes: receptor and stimulus. Associated with each connector type, 
is a set of possible values the connector can assume. As a receptor, the connector is 
capable of connecting with a connector of a matching type that is extended in stimulus 
mode. That is, the connector is capable of attaching to a signal (molecule) released by an 
agent or object in the system. In addition to the basic actions, connectors may be 
augmented with type-specific actions. 
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Connector c = <h,fc, s, m, t, V,v, A> 
h - host 

fc - control function 

s - state; s e {extended, retracted, connected} 
m - mode; m e (receptor, stimulus} 
t-type 

V- set ofpossible values for connector type t 
V -current value (ve V) 

A - action set; 

A = {extend, retract, connect, disconnect, type-specific actions} 

Equation 4, Connector Deflnition 
a. Graphical Notation 

Before proeeeding further, it is necessary to briefly describe the graphical 
notation developed in [VanPutte, 2002] for visualizing connectors. The notation was 
developed to depict a specific type of connector (iconnector) and has been modified 
somewhat to present a general definition of connectors. Figure 14 depicts three 
connectors in various states; retracted, extended (stimulus) and extended (receptor). The 
solid lines extending to the left into environment E indicate the source of the input to the 
respective control functions. All three connectors are receiving input from various parts 
of environment E. The symbol to the right side just below the connector arm indicates 
the connector type. Connector W is of type y and connectors X and Y are type p. 
Connector W is shown in a retracted state, while X and Y are extended. When extended, 
connectors operate in one of two modes, stimulus or receptor as indicated by the different 
ends on connectors X and Y. The symbol in the circular end of connector X indicates the 
value of the connector. The receptor on connector Y is labeled with the set of values it 
can connect with (a and c); in the case where it can connect with any value, it is labeled 
with an 
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Figure 14, Graphical Depiction of Connectors 
b. Host Entity 

Connectors are attached to objects and agents in the simulation. The 
object or agent to which they are attached is called the host entity. As such, connectors 
become an extension of the host and function to describe the host’s state. The host entity 
and connector control function are tied closely together in that the host entity serves to 
define the scope of information available to the connector’s control function. That is, the 
control function operates primarily on input from the host. However, when the connector 
is extended, information received via the connector is also available. Figure 15 shows a 
connector attached to a host agent. The control function is receiving input from the 
agent’s inner environment. With the connector extended into the outer environment in 
receptor mode, any stimuli received by the connector are also fed to the control function. 
Connectors are also associated with an agent modeling construct called tickets, which is 
described later in this chapter. 


59 







Agent 



Figure 15, Connector Attached to a Host Agent 
c. Control Function 

The control function (/^) manages the connector’s state changes, as well as 
value changes, and controls execution of domain-specific actions. The term behavior is 
used in the context of connectors to describe “the execution of state changes, value 
changes and domain-specific actions relative to the state of the host to which it is 
attached.” 


The control function defines the behavior of the connector in terms of 
domain-specific measures and values. For instance in the case of state changes,/c defines 
precisely, based on the information available to the connector, when the connector 
changes state. Figure 16 shows two agents from a predator/prey simulation where both 
agents are equipped with food connectors. 
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Predator 



food stimuli retracted 
or 

hunger < Mq 

Figure 16, Predator/Prey Control Function 

The predator’s inner environment ineludes a state variable that eaptures its 
hunger level. The eontrol function of the food connector is linked to this hunger state 
variable. As the hunger level increases above a threshold of hi, the connector extends in 
receptor mode in an attempt to locate food. If the predator senses the presence of food 
via a food stimulus, then a connection is established. Because of the connection, the 
predator is able to satisfy its hunger. The connection is broken once the prey’s food 
source is exhausted (connector retracts), or the predator’s hunger level is reduced below a 
threshold of ho. 
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d. State 

A connector has three possible states: extended, retracted and connected. 
In addition, an extended connector has two modes, receptor and stimulus. The control 
function manages the state transitions as depicted in Figure 17. 



Figure 17. Connector State Transition Diagram 

(1) Retracted. In a retracted mode, the connector is inactive with 
respect the environment outside of its host entity. The control function continues to 
receive updated information from within the host, and uses this to change the connector’s 
state. From a retracted state, the connector is able to transition to an extended state, 
either in stimulus or receptor mode. 

(2) Extended. When a connector extends, it does so in one of two 
modes: receptor or stimulus. A full description of the two modes is provided below. 
When in an extended state, the connector has access to the environment outside of its 
host. In the stimulus mode, the connector is broadcasting its value to all connectors 
extended in receptor mode capable of sensing the stimulus. In receptor mode, the 
connector senses the environment for stimulus from connectors of a compatible type. 
From an extended state, a connector can transition to a retracted state, or alternatively, to 
a connected state. 

(3) Connected. Connectors extended in receptor mode are capable 
of establishing a connection via a connect action. A connection opens a conduit between 
the connected entities through which high-level interactions are possible. Examples 
include pushing and pulling data, increasing or decreasing an agent’s resource levels or 
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adding procedural knowledge to an agent to simulate an increased level of training. In 
[VanPutte, 2002] once an agent establishes a eonnection with piece of infrastructure, the 
agent ean “puh” information (i.e. aeeess codes and passwords), push information (i.e., a 
virus), or directly modify the entity (wipe the hard-drive). As ean be seen, the action set 
is specific to the conneeted entities. The type and seope of action is application 
dependent. 

Connectors in a connected state ean break the eonnection with a 
retract action and transition to a retracted state, or exeeute a disconnect aetion and return 
to an extended state in reeeptor or stimulus mode. 

e. Mode 

When eonneetors extend, they do so in one of two modes: stimulus or 
receptor. In stimulus mode the connector takes on a single value from its set of possible 
values and broadeasts its type, value and host entity to the environment. From the 
stimulus perspective, the information is sent with no specific intended target. 

Connectors extended in receptor mode are essentially in a listening mode. 
They listen for, or sense, connectors in their environment of the same type that are 
broadcasting in stimulus mode. Ineoming sensory information, in the form of value and 
host entity, is passed to the control function for processing. 

While in receptor mode, it is possible for the connector to enter into a 
connected state with another connector. The receptor is labeled with a set of values from 
the connector’s value set that the receptor is eapable of connecting with. This set is 
dynamic in that the eonnector’s control function continually updates the members. Onee 
connected, additional high-level interaetions are possible between the eonneeted entities. 
These “high-level” interaetions are defined by the host entities. The eonneetors simply 
open up the communieations channel between the hosts. 

Equation 5 defines a current value function (O) that returns the set of 
values the reeeptor can connect with, or the value of the stimulus. 
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Given, 


c = <h,fc, s, m, t, V, V, A> 


Vc ((s = extended) & (m = receptor)) 

0(c) = ^ 

V e V ((s = extended) & (m = stimulus)) 

undefined otherwise 


(Vc = the set of values labeling receptor c) 


Equation 5, Connector Current Value Function (^)) 

f. Type 

Connectors are defined in terms of their type. Only connectors of the 
same type can connect with one another. The type designations for connectors are 
application dependant; there are no predefined types. In the story engine instance 
described in Chapter VII, character agents are defined with seven personality traits and 
six resource categories, with a corresponding connector type for each. This finite number 
of traits is based on the particular application - Army careers/values - and is not a 
limitation of the story engine. Defining the connectors based on personality and 
resources establishes a character-to-character communications protocol structured in 
terms that are important to Army career progression stories. 

g. Value 

For each connector type, there is a defined range of possible values. 
When a connector is extended in stimulus mode, the connector’s current value is 
broadcast along with the type and host entity. Much like type, values are application 
dependent with no predefined values for connectors. In the story engine characters 
described above, one of the resource categories is Energy. For the Energy type 
connector, there is a set of five possible discrete values ranging from Low to Eligh (Figure 
18). In the example, the connector value is from a discrete set but there is nothing to 
prevent the connector value from being the output of a continuous function. 
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Figure 18, Character Agent Energy Connector 
h. Action set 

The action set defines the actions that the connector’s control ftinction can 
initiate. There are four basic actions required of a connector: extend, retract, connect and 
disconnect. In addition to the four basic actions, connectors may be extended with type- 
specific actions appropriate for the given simulation domain. The basic actions are used 
to transition the connector between its states (Figure 17). 

Extend and retract expose and rescind the connector arm as previously 
described. If the connector is in a connected state, then retract breaks the connection. 
Disconnect breaks the connection between two connectors, but leaves the connector 
extended. Connect initiates a connection between two connectors provided all of the 
prerequisites have been met. Equation 6 defines the predicate conn(ci, C 2 ), which 
describes the requirements for two connectors to connect. 
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Given, 

Cl = <hi,fci, Si, mi, ti, Vi, vj, A]> 
and 

C2 = <h2,fc2, S2, m 2 , t2, V 2 , V2, A2> 
then Cl can connect to C 2 (conn(ci, C 2 )) 

iff: 

Si = S 2 = extended 
mi = receptor & m 2 = stimulus 
V 2 e 0(ci) 

_ tl = t2 _ 

Equation 6, Requirement for Two Connectors to Connect - conn(ci, C 2 ) 

At a minimum, for a connection to be established, the reeeptor and 
stimulus conneetors must be of compatible types and the stimulus value must match the 
label on the receptor. Once these minimums are met, the control function may execute a 
connect action binding the two entities together. It is important to note that the 
connection is not necessarily automatic. Even though a connection is possible, must 
still initiate the connect action. As an example, in the case of iconnectors from 
[VanPutte, 2002], is directed to initiate the conneetion by a higher level mechanism 
called an ibinder. In the story engine CMAS described in Chapter VI, conneetions are 
initiated locally by the agents. 

F. TICKETS 

One of the major benefits of agent-based modeling is the innovative and adaptive 
behavior agents bring to simulations, however, it is often desirable to balance this with 
doetrinally eorrect and appropriate actions. Symbolic Constructor Agents and the goal 
apparatus were developed to eontrol the agent’s sensory eapability and decision-making. 
In order to provide agents with a rieh proeedural knowledge base while still supporting 
adaptive behavior, a data structure called tickets has been developed. Tickets allow 
agents to apply proeedural knowledge in context. They define the agent’s action set, i.e., 
its means to achieve its goals. They are used to organize proeedural knowledge and 
provide the ability to balance doetrinal behavior with adaptive, innovative action, 
resulting in enriehed problem solving behavior. 
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Tied to each of an agent’s goals are one or more tickets that define how to achieve 
the goals (Equation 7). Tickets are defined by a control function (f.), a set of connectors 
(Q, and a set of frames (F). 

Ticket = <fc, C, F> 
fc - control function 
C - connector set 
F - {frame 1 , frame 2 , ..framej; 
framei e {action, ticket, reference} 

Equation 7. Ticket Definition 

1. Ticket Control Function 

The control function has four jobs. When the ticket is first executed, it performs 
any necessary initialization. Second, f controls the sequence and manner of frame 
execution. Since f is user defined, any execution sequence is possible, including 
sequential, random, looping, etc. In addition, the tickets can be defined so they execute 
through their entirety as a single action (execute all frames without interruption), or they 
can execute a single frame each time it is the agent’s turn to act (single step). Third, 
when the ticket is complete,^^ resets the ticket as appropriate. Finally, it coordinates with 
the attached connectors to initiate their connect actions. 

2, Ticket Frames 

Tickets are comprised of a set of frames, F. A frame can be thought of as a 
container that holds a procedural step. However, simply encoding step-by-step 
procedural knowledge and linking it to various goals is not sufficient for creating 
intelligent behavior. The desire is to apply the most appropriate actions or procedures for 
the “given situation.” In a dynamic system, the “given situation” not only changes 
constantly, but is often so complex the system designer cannot conceive of, much less 
account for, every possibility. Therefore, the mechanism for determining the “most 
appropriate” procedures must be flexible and able to support the same level of 
complexity as the changing contexts of the dynamic system. By combining connectors 
with tickets, the desired flexibility can be achieved. Figure 19 is a graphical depiction of 
a connector-based ticket. 

Ticket frames can be static or dynamic [Hiles et al., 2001], [VanPutte, 2002]. 

Static frames hold a “by-name” call for a specific ticket or action, while dynamic frames 
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hold a more general “referenee” for the type of ticket or action. By-name specification of 
tickets and actions equates to hard-coding behaviors at design time. In many cases, 
constraining an agent’s action in this way is desirable. In Figure 19, frames one and three 
contain by-name calls to Action X and Ticket A respectively. 

Ticket frames can also hold ticket and action references. These references allow 
the designer to specify the general nature of the behavior, but delay the final binding of 
the specific action or ticket until run time. 

References are simple lightweight objects that coordinate the actions of 
connectors extended in receptor mode. When it comes time for a dynamic frame to 
execute, the reference object initiates a connection with an appropriate ticket or action. 
This run-time binding ensures the behavior will be appropriate for the given situation as 
perceived by the agent. A detailed description of references is provided with the 
discussion of Composite Agent actions in Chapter V. 

Returning to Figure 19, frame two contains a reference for an action described by 
a type a connector with value r and a type P connector with value s. These extended 
connectors (type a and P), as well as their respective values (r and s), were not fixed at 
design time, rather they result from the current state of the host agent. They are not 
bound until the reference is evaluated for execution at run-time. In this example. Action 
Y meets the criteria and will be executed in frame two. Similarly, frame four contains a 
reference for a ticket with a type a connector with any value (*). Ticket B will be 
executed at frame four. 
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Figure 19, Example Connector-based Ticket 
3. Ticket Connectors 

Tickets have prerequisites that must be met in order to activate. The prerequisites 
are defined in terms of conneetors and are specific to each ticket. Figure 20 shows a view 
of a eomposite agent with the connectors’ control functions receiving input from the 
agent’s inner environment. Pereeptions from the outer environment are proeessed by the 
SCAs which in turn update Einner- Connectors attached to the tickets are likewise 
updating based on the agent’s inner environment and dynamically change state as Einner 
ehanges. In Eigure 20, Action Xhas the neeessary eonneetors extended in stimulus mode 
that will allow it to connect with the referenee in frame one of Ticket A. 
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Figure 20. Ticket and Action State Dynamically Maintained with Connectors 

With the connectors continually reacting to the environment, behavioral and 
procedural knowledge (tickets and actions) can bind at run-time to fit the context as it 
develops. This binding is based not only on the state of the environment, but also on the 
goals of the agent, its capabilities, and its social interactions with other agents. In this 
way, the correct procedural knowledge can be brought to bear in the appropriate 
situation. 

G. SUMMARY 

In this chapter, the concept of semi-fluid software structure was introduced. 
Connectors and tickets were formally defined and the biological inspiration behind 
connectors was presented. When connectors are combined with tickets and actions, 
reference actions can be used to bind actions to tickets at run-time. This run-time binding 
provides agents with the ability to execute contextually appropriate actions in pursuit of 
their goals. Connectors, tickets and composite agents form the foundation for a 
Connector-based MAS (CMAS) architecture. The following chapter defines the CMAS 
architecture and develops a simulation model that makes extensive use of connectors and 
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the process of connecting, by which agents employ a simulation specific “best-fit” 
algorithm to bind to other agents. 
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V. CONNECTOR-BASED MULTI-AGENT SYSTEM 

ARCHITECTURE 


A, INTRODUCTION 

This chapter introduces the Connector-based Multi-Agent System (CMAS) 
architecture. A formal definition of a CMAS is presented, along with an architecture for 
a connector-based composite agent. The CMAS architecture and connector-based agents 
form the foundation for the story engine. 

B, CONNECTOR-BASED MULTI-AGENT SYSTEM 

One of the difficulties in modeling complex systems is defining the function 
described in Equation 2 that reduces the set of all possible actions that an agent can take 
to a set of actions that are appropriate for the current situation {Oa,t). The function is 
far too complex for a top-down design. The decision of what to do next, based on all of 
the possible things that can be done, must be pushed down to the local level where the 
agent decides. In addition, once the decision is made to act, it is preferable to not simply 
take any action that will work, but act in a manner that is most appropriate to the given 
situation. In order to take the appropriate action, binding the specific action should be 
delayed as long as possible. 

Connectors provide a general modeling construct used to facilitate 
communications and assist in the delayed binding of actions. Their design makes them 
an excellent mechanism for representing the “current situation,” and when combined with 
tickets, they provide the ability to express “what should be done” with the flexibility of 
delaying the decision of “how it should be done.” 

In a connector-based MAS, the environment is populated with agents and objects 
that express their state through extended connectors. The extended connectors provide a 
sensory stream for other agents in the simulation. Through an operation of connecting, 
agents interact with one another and objects in the environment. Connecting employs an 
application specific “best-fit” algorithm to bring entities together and facilitate their 
interaction. These connections are active for a finite period of time during which the 
connected entities interact through the communication channels opened up by the bound 
connectors. Through a continual process of connecting and disconnecting, the system 
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evolves (Figure 21). In the ease of the story engine, this proeess results in stories being 
generated one eonneetion at a time. 



Figure 21, Connector-based MAS 

Equation 8 defines a MAS arehiteeture based on the fundamental behaviors of 


eonneetors and tiekets. 

MAS = {E, O, A, R, Op, Laws, Q 

E - Environment = <A, O, C> 

O - Objects situated in the environment 
A - Connector-based agents 
R - Relations linking agents and objects 
Op - Operations 

Laws - Laws governing the environment 
C - Connectors 

Equation 8, Connector-based Multi-Agent System (CMAS) Definition 
C. ENVIRONMENT 

As with a MAS defined by Equation 1, a CMAS may be situated or non-situated. 
The environment is populated with agents and objeets, and assoeiated with eaeh of these 
entities are eonneetors that deseribe the entity’s eurrent state. In a CMAS, the agents do 
not direetly pereeive the other agents and objeets in the environment; they sense the 
eonneetors that are exposed to the environment by the entities. Therefore, while 
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Equation 9 includes agents and objeets in the environment, they only exist through their 


eonneetors. 

E = <A, O, C> 

A - Agents 
O - Objects 
C - Connectors 


C = c(A) uc(0) 

where c(x) is the connector set associated 
with entity x; x e A or x e O 

Equation 9, CMAS Environment 

D, OBJECTS 

Objeets in a CMAS are non-agent entities in the environment. They ean be 
pereeived, ereated, destroyed and modified by agents. Unlike Ferber’s definition of a 
MAS, agents are not a subset of the objeets. The primary differenee between agents and 
objeets is that agents are aetive entities eapable of taking aetion based on their own intent. 

Objeets are differentiated by type, and within the type, further differentiated by 
speeifie instanees of the type. For example, roeks, trees and radio transmitters are objeet 
types. For eaeh of these types, there are instanees with attributes speeifie to the instanee. 
In the ease of radio transmitters, broadeast frequeneies might be used to differentiate 
transmitter instanees. 

CMAS objeets depieted in Figure 22 and defined by Equation 10 eonsist of 
eonneetors representing their state, and a eontrol funetion that allow the objeets to ehange 
state based on interaetions with agents in the environment. In the ease of objeets, the 
eontrol funetion fc is basieally a state maehine; on input x, the objeet reaets in a 
predietable way, resulting in new state y. 
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Object 



Figure 22. CMAS Object 

o = <fc, C> 

fc -function controlling the object’s state 
C - set of connectors defining the state of object o 

Equation 10. CMAS Object Definition 

In general, the set O is defined as the union of all of the possible object types for 
the system (Equation 11). 

If there are n unique object types in the system, and 
Oi = instances of object type i 
then 

0=[joi ; i =l,..,n. 

i 

Equation 11. CMAS Object Set (O) 

Since objects can be created and destroyed during the life of the system, to be able 

to evaluate the state of the objects as a whole, O must be defined relative to the specific 

object instances at time t (Equation 12). 

If there are n unique object types in the system, and 
Oit = {instances of object type i in the system at time t} 

then 

Ot=\JO^,;i = I,..,n 

i 

Equation 12. CMAS Objects at Time t (Ot) 
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E, CONNECTOR-BASED COMPOSITE AGENTS 

Connector-based agents are agents that adhere to the use of connectors as a 
primary communications mechanism and means of expressing their state to other agents 
in the environment. Many classes of intelligent agents can be employed in a CMAS; the 
only defining factor being that they interface with the outer environment through 
connectors. They do not have to employ connectors as their sole interface, but in as 
much as the agent wants its state to be known by other entities in the simulation, it does 
so through connectors. This section describes connector-based agents in terms of the 
composite agent architecture described in Chapter IV. 

The previous chapter defined connectors and described their behavior. This 
section describes how to put connectors to work to create a flexible communicating 
agent. Equation 13 defines a connector-based composite agent. 

Cl ^^50 T, £2, C/. G, 

Asc - set of symbolic constructor agents 
T- tickets 

£2- agent’s action set 
Einner- agcnt’s inner environment 
Ci - local set of connectors internal to agent a 
Ce - connectors used to externally portray agent a’s state 
G - set of goals for agent a 

Ar - set of reactive agents to manage agent a’s goal set 
Equation 13, Connector-based Composite Agent Definition 

Composite agents employ a sense-update-act model where they sense their 
environment, update their internal view of the world, then decide what action(s) to take. 
The agent’s sensory capabilities are defined in terms of its SCAs and extended 
connectors, which maintain an internal representation (Einner) of the agent’s view of the 
world. Erom this Einner view, the agent employs a dynamic goal structure managed by 
reactive agents (RAs) to act in accordance with contextually appropriate procedural 
knowledge (tickets). In the course of their sense-update-act cycle, the agents make 
extensive use of connectors to maintain the state of their inner environment (Ci), select 
and take action, and express their state to the outside world (Ce). Eigure 23 depicts a 
single composite agent processing sensory input from the outer environment. The 

following sections describe the connector-based composite agent in detail. 
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Figure 23, Connector-based Composite Agent 

1, Symbolic Constructor Agents and Emner 

Symbolic constructor agents (SCAs) process sensory input from the outer 
environment (Eouter) and manipulate it as neeessary to eonstruct and maintain the agent’s 
inner environment (Einner). SCAs receive their stimuli by any available means; this may 
include the use of connectors, but it is not required. Depending on the particular domain 
being modeled, it might not be suitable for all external sensory information to be 
eonneetor-based. 

The architecture of SCAs is domain dependent and elosely integrated with the 
structure of the inner environment. They can be simple reaetive agents or complex 
cognitive agents using symbolic reasoning to interpret and correlate sensory streams to 
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build high-level representations of the input. SCAs ean take advantage of any desired 
means to process sensory input and construct an inner representation. 

As difficult as it is to define a precise architecture for SCAs, it is just as difficult 
to develop an exact definition of what comprises an inner environment. In general, Einner 
can normally be described in terms of internal connectors, state variables and objects 
(Equation 14). 

Einner = <Ci, Sv, 0> 

Ci - set of connectors internal to the agent 
Sv - set of domain specific state variables 
O - Objects 

Equation 14, Agent Inner Environment (Einner) 

The connector set C provides an intra-agent coordination and control capability. 
These connectors are identical to external connectors, with the exception that internal 
connectors never communicate directly with the outer environment. 

State variables provide a measure of agent characteristics that are best captured 
through a numeric value. These values are updated through the SCAs and agent actions. 
Internal and external connectors often use these state variables as input to their control 
functions. 

The objects of Einner follow the definition from Equation 10. Their presence and 
state is broadcast via their extended connectors. Just as agents in the outer environment 
can connect to objects in their environment, SCAs and RAs can likewise connect to 

objects from Einner- 

2, Tickets and Actions 

Composite agents act on their inner and outer environments through actions 
selected by a dynamic goal structure. Connector-based composite agents make use of 
connectors, in conjunction with a goal structure, to control and coordinate the application 
of both tickets and actions to achieve complex behavior. Tickets, by their design, make 
extensive use of connectors (Equation 7). Their function with respect to connector-based 
agents is the same as described in section IV.E. 

Run-time binding of tickets and actions is accomplished through a reference 
action. References were introduced in section IV.E with the description of static and 
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dynamic ticket frames. The following section formally defines a reference as an aetion 
speeific to a connector-based agent and connector-based multi-agent simulation. 
a. References 

References are a speeial type of aetion that ereate a eonneetion with a 
eontextually appropriate tieket or aetion. Using eonneetors extended in reeeptor mode to 
eapture the eurrent state (or eontext) of the agent, the reference eonneets with an aetion or 
tieket whose function is described by eonneetors extended in stimulus mode. Equation 
15 defines a reference in terms of its eontrol function and connector set. 

Reference r = <f, Cr> 

fr - control function 

Cr- r’s set of connectors 

Equation 15. Reference Definition 

The function f manages the extension and retraetion of the eonneetors 
from Cr based on input from Emner- In addition, for the extended eonneetors C(- c Cr,fr 
determines the set of values eaeh of the eonneetors from C'r can connect with. Eor 
instance, if c e C ^ is a eonneetor extended in reeeptor mode, then c is able to establish a 
eonneetion with another eonneetor whose value is a member of the set 0(c). Working in 
eonjunction with eonneetor c’s eontrol funetion,/- adds or removes values from the set. 
In this way, the eonneetors extended by the reference are always attempting to eonneet 
with notions and tiokets that are appropriate to the eurrent state of Einner- Equation 16 
defines the oonditions under whioh a reference will eonneet with a tieket. References 
eonneet to actions in a similar manner as desoribed in the next seotion (Equation 18). 

Given, 

Reference r = <fr, Cr> 

Ticket tk = < fk, Ctk, F> 
where, 

C'r ^Cr is r ’s set of extended connectors (receptor mode) 

C'tk ^ Ctk is tk’s set of extended connectors (stimulus mode) 

then r connects with tk iff: 

(Vcre C'r)3ctke C'tk (conn(cr, Ctk)) 

(conn(Cr, Ctk) defined by Equation 6) 

Equation 16. Reference Action to Ticket Connection 
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b. Actions 

Agents interact and modify their inner and outer environments by 
executing actions. In a connector-based agent, actions are made up of three components 
(Equation 17); a control function (fa) used to coordinate the connectors, a set of 
connectors (C^), and finally the underlying action that is executed when the action 
wrapper executes. 

Action a = <fa, Ca, act> 

fa - control function 
Ca - set of connectors (Ca ^ Ci u Ce) 

act - underlying action 

Equation 17. Action Definition 

References, as defined above, are also capable of connecting with actions. 
Equation 18 defines the conditions for the connection. 

Given, 

Reference r = <fr, Cr> 

Action a = <fa, Ca, act> 
where, 

C'r cCr is r’s set of extended connectors (receptor mode) 

Ca^Ca is a’s set of extended connectors (stimulus mode) 

then r connects with a iff: 

( VCr e C'r )3Ca ^ C a (cOnn(Cr, Ca)) 

(conn(Cy, cj is defined by Equation 6) 

Equation 18. Reference Action to Action Connection 

Actions may require certain resources in order to execute. The resource 
requirements are captured in the control function. Through a connector’s connect action; 
an action can bind to an object to fill its resource needs. Eigure 24 shows an example 
where an EAT action connects to a FOOD object resulting in a connector extending on 
the action which allows it to connect to the reference. 
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Figure 24, Eat Action Binding to Food Object 

c. Signal Cascades 

Signal cascades, as described in seetion IV.E.l are a sequence of aetions 
and state ehanges that oeeur in a eell as the result of external stimuli. When the internal 
eomponents of a eell are aligned in just the right states, a single stimulus can trigger a 
eomplex sequenee of internal aetions (Figure 13). 

Through the eombined use of conneetors, referenees, tiekets and aetions, 
the power of signal eascading ean be simulated in agents. Figure 25 depiets a caseade 
that is initiated when agenf eonneets with agentj. In this example, the external 
eonneetion triggers an a signal being sent to referenee W. W in turn updates the label set 
on its extended reeeptor allowing it to eonneet with tieket P. The first frame of P 
exeeutes aetion A; the seeond exeeutes referenee X, resulting in execution of action F. 
The third frame exeeutes referenee Y that binds to tieket Q, resulting in the eventual 
exeeution of aetions F and B. 

If the internal state of agentj had been different, it is possible that the 
eonneetion would have triggered a different set of aetions, or agentj may have failed to 
eonneet with agenti all together. 
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Figure 25, Signal Cascade 

3, Agent Connector Sets 

An agent’s connector sets (C, and Ce) form the backbone of its intra-agent and 
inter-agent communication, coordination and control system. Virtually every aspect of 
the agent is influenced in some way by connectors. The set C, is defined locally within 
the agent and is used strictly in an intra-agent role to meet the specific agent’s needs. The 
scope of these connectors does not exist beyond the agent. 

Defining the origin of Cg is more difficult. Depending of the developer’s point of 
view, Ce can be defined as a subset of the CMAS’s connector set C, (Cg ^ C). This 
implies that C is a fixed set that was defined in a top-down fashion and all external 
connectors must come from this pre-defined set. Alternatively, the CMAS connector set 
C can be defined as the union of the connector sets from all of the agents and objects in 
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the system. In this ease, C is defined in a bottom-up fashion and eould eoneeivably 
ehange over time. Considering the biological inspiration behind connectors, evolutionary 
theory would lead us to the conclusion that the connectors adapt and evolve locally to 
meet the needs of the host organism. This is fine in theory, but when it comes to building 
connector-based software systems, allowing connectors to evolve has not yet been 
explored. 

In the connector-based architecture presented here, it is assumed that Ce is defined 
at the agent level, but connectors do not evolve, and the CMAS connector set C is the 
union of the connector sets defined by the objects and agents. In this way, when new 
agents and objects are introduced to the system, it is possible to include their connectors 
in the CMAS set C (Equation 27 later in this chapter). 

4, Reactive Agents and Goals 

The behavior of an agent can be thought of as the manifestation of the actions the 
agent takes in pursuit of its goals. As described in Chapter IV, the reactive agents (RA) 
of a Composite Agent function to control the agent’s behavior. For any single behavior, 
there may be multiple goals responsible for producing the behavior. In the Composite 
Agent architecture, RAs are defined with the intention that a single RA be responsible for 
a single behavior (Equation 19). 

ClR = <fR, Gr> 

/r - control function 
Gr - goal set for qr ; 

Equation 19. Reactive Agent Definition 

With behaviors emerging from multiple goals, ur is basically a goal management 
apparatus where fR manages goal set Gr. Gr is the goal set for a single RA; the 
Composite Agent’s goal set G is defined in Equation 20 as the union of the goal sets for 
each of the reactive agents. 
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Given agent a with reactive agent set Ar, 
if Gk = goal set for reactive agent aR(k) e Ar 

then 

G=[^G,;k = l,..,\AR\ 

k 

Equation 20. Agent Goal Set (G) 

Equation 3 from Chapter IV defines goals as being eomprised of a state, 
measurement method, weight and set of notions. This definition is modified slightly for 
oonnector-based composite agents. For connector-based agents, the “set of actions” is 
refined to the point where a goal can take one of three possible actions. First, it can 
execute a “by-name” call to a specific action from the agent’s action set. This obviously 
results in execution of the designated action. Second, a “by-name” call can be made to a 
specific ticket. In this case, the ticket is executed as described in Chapter IV, eventually 
resulting in a single action or series of actions being performed. Finally, a reference can 
be executed which results in the run-time binding and execution of a contextually 
appropriate ticket or action. References allow the active goal to select an action that not 
only helps achieve the goal, but is also appropriate for Composite Agent’s current state. 
In other words, it results in context sensitive action selection. When it comes time for the 
agent to act, the reference from the active goal is evaluated resulting in a connection to 
one the Composite Agent’s tickets or actions (Figure 26). 
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Reactive Agent (RA) 



Figure 26, Reactive Agent Goal Structure 

The architecture of a Composite Agent is intended to accommodate individual 
agents with a tremendous range of behaviors. However, even with a single RA and small 
set of goals, a tremendously complex range of behavior is possible. 

F. OPERATIONS 

Ferber describes operations as actions that make it possible for the agents to 
perceive, produce, consume, transform and manipulate the objects and agents in the 
environment [Ferber, 1999]. There are three actions that are unique to connector-based 
multi-agent systems, and set CMASs apart from other agent-base architectures. The first 
is the fundamental connector level action of connect. Equation 6 defined the conditions 
under which two connectors are able to connect. The reference action (Equation 15) was 
constructed from connectors to provide agents with the capability to select and execute 
contextually appropriate tickets and actions. The final action, called connecting or a 
connection action, allows agents to interact with other agents or objects. All three of 
these CMAS unique actions, connect, reference and connecting, are assumed to be part of 
every agent’s action set. Erom this description, the following definition for Operations is 
derived (Equation 21). 
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Given CMAS M with agent set A, 

M’s operations set O is defined as: 

Q =U^(«,) i = \A 

i 

at e A 

Gl{a .) = action set for agent ai 

Equation 21. CMAS Operations Set (Q) 

1. Connecting 

Connecting is the process by which agents become bound to, and interact with, 
other agents and objects in the environment. By connecting, agents establish a 
connection and are said to be connected. During a connection, or while connected, 
agents interact and communicate with the bound entities. Connections are established 
and maintained across a set of connectors. The bound connectors provide the 
communication channels for the connected entities. The entities engaged in the 
connection have the full range of communication and coordination actions provided by 
the bound connectors at their disposal. Section IV.E.2 provides a detailed description of 
connector actions. 

Equation 22 describes the conditions under which two agents are able to connect. 
A connection can be established between agenf and agentj if there exists a subset of 
connectors from agenf’s set of extended connectors (receptor mode) such that for 

every Ce(i) £ C"e{i) there exists a matching connector on agentj extended in stimulus mode. 
Individual connectors match if they meet the criteria established by the predicate 
conn(ci, Cj) (Equation 6). Equation 23 is a similar equation that describes the conditions 
under which agents connect to objects. 

Given agents at and aj with external connectors sets Ce(i) and Ce(j) respectively, 

if 

C e(i) c Ce(i) are extended in receptor mode 

C e(j) c Ce(j) are extended in stimulus mode 

then ai is able to connect with aj (conn(ai, af) iff: 

(3C e(i) ^ C e(i) )(3Ce(i) ^ C )3Ce{j) S C e(/) (conn(c e(i), Ce(j))) 

(conn(Ce(i), Ce(j)) is defined by Equation 6) 

Equation 22, Agent-to-Agent Connection Conditions - conn(ai, aj) 
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Given agent at with external connector set Cefi) and object oj with connector set Cj, 

C 'e(i) ^ Ce(i) are extended in receptor mode 
Cj^Cj are extended in stimulus mode 

then Oi is able to connect with oj (conn(ai, oj)) iff: 

(BC"e(i) ^C'e(i))(VCe(i)e C"e(i))3Cje C'j (conn(Ce(i), Cj)) 

(conn(Ce(i), Cj) is defined by Equation 6) 


Equation 23. Agent-to-Object Connection Conditions - conn(ai, Oj) 

Equation 22 and Equation 23 establish the criteria under which it is possible to 
connect. The actual decision to initiate a connection comes from one of two sources. 
Connections are initiated internally by an agent or externally by a higher-level control 
mechanism in the simulation. In [VanPutte, 2002], an external mechanism called an 
ibinder is used to bring together agents and objects with matching iconnectors. Agents 
and infrastructure objects register with the ibinder. When resources or vulnerabilities are 
exposed via connectors by the infrastructure objects, the ibinder is notified. Eikewise, 
the ibinder is aware of the resource requirements of the agents. The ibinder serves as a 
digital switchboard, connecting agents to resources. 

The story engine, which is described in the next chapter, strictly uses agent- 
initiated connections to generate stories. Agent-initiated binding is a self-serving process 
where agents look for connections that are advantageous to themselves and promote their 
current goals. This does not mean the agents cannot or will not cooperate. On the 
contrary, connections can be weighted so that mutually beneficial connections are more 
likely to occur. 

The agent has the following information available when evaluating the fitness of a 
connection: its internal state, its goals with their status and measure, and the type and 
value of the connectors it is evaluating for connection. In addition, conn(ai, aj)^ being 
true guarantees that the connection is valid with respect to the laws of the story world. 
Erom the set of possible connections the agent can engage in, it must determine which 
one appears to be the most beneficial based on its goals and current state. 


3 The conn predicate provides a means of formally describing the conditions under which a connection 
may occur. It is not meant to imply that connector-based composite agents, or any connector-based agent 
must use formal logic methods to manage their connection process. 
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At any given time, an agent is aware of a set of entities (objeets and agents) in its 
environment. This set of entities W, is deseribed as the agent’s awareness set (Equation 
24). 

Given 

CMAS M with agent set A, object set O, 
and 

agent a e A, 

HW^AuO, 

s.t. a is aware of the agents and objects ofW. 

Agent a maintains W through its extended connectors and SCAs. 

Equation 24. Awareness Set (W) 

Given a with awareness set W, there is a set W'^ W that a is able to eonneet with 

(Equation 25). W'is known as the agent’s set of candidate connections. Note that it is 

possible for W'= 0'm whieh ease the agent eannot engage in a eonneetion. 

Given agent a with awareness set W, 
a’s set of candidate connections is defined as: 

W' = \^conn{a,e) 

e^W 

Equation 25, Candidate Connection Set (W') 

Equation 26 defines an evaluation funetion for eomparing the respeetive value of 

eaeh eonneetion when multiple eonneetions are possible. 

Given agent a with external connector set C, 
goal set G, and candidate connections W 

(VeeW') 3C'cCfor which conn(a, e) is true. 

C'is the set of connector(s) over which a can connect to e. 

The most favorable connection (mfc) over the set W'is 

mfc = maxf(C'i, G, <j(a)); i = 1,..,\W] 

i 

^(a) is an evaluation of agent a’s state, 
f is a domain specific evaluation of the value of connection i. 

Equation 26, Most Favorable Connection (mfc) 

G. RELATIONS 

In a previous ehapter, relations were deseribed as “abstraet links” that ereate a 
dependeney between the agents. Conneetors provide a natural means for establishing and 
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maintaining relationships between agents. They ean be used to establish private networks 
between agents. The “related” agents ean then eommunieate and eoordinate over the 
private network. 

An agent’s set of extended eonneetors defines its seope of interest and infiuenee. 
By their type and value attributes, they define the other agents and objeets in the 
environment the host agent is interested in interaeting with. This seope of interest and 
infiuenee is preeisely what is at the heart of relationships. The “abstraet links” used to 
deseribe relations have a one-to-one eorrespondenee with eonneetor types. When a group 
of agents shares a set of eommon eonneetors, they have the ability to interact as a 
coordinated group over a “private” communications network. The level and type of 
coordination is determined by the “relationship-specific” actions the related agents 
possess. 

The definition of connectors allows for the basic action set of a connector to be 
augmented with “type-specific” actions. When a set of relationship-specific connectors is 
defined, these type-specific actions are used to implement actions that are unique to the 
relationship. 

Figure 27 shows a CMAS environment populated with agents and resources. 
There are three types of resources (r, s, and t). Each agent has a primary goal of 
collecting as many resources as possible, but only of a single type. Connector type P is 
used to sense and bind to resource objects. Agent D, for instance, can sense and collect 
resource r. The agents have a limited perception range and randomly explore the 
environment in search of resources. Agents A, B, and C have established a cooperative 
relationship whereby they share information and notify each other when they locate a 
resource of interest to one of the other agents. The relationship is established through a 
type A, connector. By entering into this relationship, the agents are able to sense all three 
resources, not just their primary resource. When an agent detects a resource that is of 
interest to another agent in the relationship, it extends a type A, connector in stimulus 
mode. The interested agent establishes a connection through which the resource type and 
location are passed. This relationship greatly extends the related agents’ sensory range. 
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Figure 27, Resource Sharing Relationship 

While this example is relatively small, it serves to demonstrate the role connectors 
play in establishing and defining relationships in a CMAS simulation. 

H. LAWS 

The laws of the CMAS are both global and distributed. They are intended to 
establish the boundaries for the simulation. They do not define specific paths or 
behaviors; those are left for the agents to discover. In an earlier chapter, the ideas of 
semi-fluid software structures and indirect solutions were discussed. Agents are useful 
for finding solutions the designers may not have considered. To do this, they must have 
the freedom to operate across a broad landscape. However, if the solution is going to be 
useful, there must be constraints on the landscape. These constraints are the laws of the 
simulation. They must be chosen carefully so as not to overly constrain the agents, yet be 
specific enough so the solutions are viable. This section does not attempt to define 
specific constraints for they are very much domain dependent; that is saved for the 
following chapters where the story engine and America’s Army: Soldiers are described. 
The remainder of this section identifies the components used to define and enforce 
constraints. 

In agent-based modeling, there is no single omniscient entity controlling the 
actions and interactions of the agents. With no central control, the enforcement of laws is 
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pushed down to the agent level. This is not to say that there are no global laws. 
Newton’s laws are a good example; they may apply to all agents and objeets in a situated 
environment involving motion. One set of equations can be used to describe the laws, 
but there is no centralized control mechanism that is responsible for ensuring each agent 
and object adheres to these laws. They are enforced by their encoding in the actions of 
the agents, i.e., if an agent at point x moves with a velocity vector of v for t seconds, it 
will end up at pointy. 

In the CMAS architecture, laws are also described and enforced through connect 
and connection actions, and in terms of tickets and references. All of these work together 
to define and enforce the laws. 

Chapter VII describes an implementation of the story engine set in the domain of 
the U.S. Army. The engine generates stories depicting a soldier’s progression through an 
Army career. In this simulation, tickets and references are used to capture the procedural 
constraints, or laws, associated with Army career progression and school requirements to 
achieve specific designations. The tickets ensure the soldier follows a legitimate Army 
career path. For example, when soldiers enlist in the Army, there are specific training 
paths based on their specialty. These include entry processing, basic training, advanced 
training and then a duty assignment. This progression is captured in tickets that ensure a 
soldier completes basic training before advanced training. It would not make sense for 
the soldier to jump directly to a duty assignment with no training. 

Within each training area, there are certain events every soldier must participate 
in as they progress. These progressions are not optional; they are specific laws that apply 
to the Army. Army basic training is made up of a well-defined sequence of training 
events that prepare a soldier for the next phase of training. The events are not optional, 
they must be completed, and usually in a fixed sequence. This sort of sequencing is 
captured in tickets. 

In conjunction with tickets, connectors and references are used to enforce 
prerequisite constraints. Continuing with the Army career progression example, at some 
point a soldier must select their next duty assignment. The action to select occurs when 
the soldier reaches the frame of their duty assignment ticket marking the end of the 
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assignment. The reference in the tieket frame is populated with eonneetors that represent 
the soldier’s skill set, experienee and performanee. The only sehools and duty 
assignments the soldier can select are those for which a valid connection is possible. If 
the soldier wants to join the Special Forces but does not possess the correct connector set, 
they will not be able to connect with the Special Forces ticket. In this way, tickets, 
references and connectors work together to enforce the laws of the simulation. The laws 
pertaining to qualifying for Special Forces are captured in the set of connectors extended 
(stimulus mode) from the Special Forces ticket. 

I. CONNECTOR SET 

The set of connectors comprising a CMAS can be separated into two categories. 
There are external connectors that are visible throughout the CMAS environment E. 
There is also a set of internal connectors that the agents use for intra-agent coordination 
and control. Equation 27 defines the two sets as the union of the corresponding set from 
each of the system’s agents and objects. 
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Equation 27. CMAS Connector Set (C) 

J. MANAGING THE COMPLEXITY OF AGENT INTERACTIONS 

Connectors serve to establish a context for the agents within their environment. 
Context is defined as “the interrelated conditions in which something exists.” [Merriam- 
Webster, 2002]. The context influences the agent’s goals and actions. For example, 
when a person is sitting in a meeting, their surroundings and the meeting itself establish a 
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context for the person that helps to determine what aetions are appropriate and whieh are 
not. It also influenoes the person’s goals. 

An agent’s eontext is established by its eonneetors. Therefore, given the agent’s 
set of extended eonneetors, only eertain tiekets and aetions are possible, either direetly or 
through references. As soon as the agent’s eonneetors ehange state, the eontext 
immediately ehanges and new aetions and tiekets are available. Agents may have 
extensive eonneetor sets that allow for huge number of states (or eontexts). The soldier 
eharaeters in the America’s Army: Soldiers have seven eore values eaeh with five levels, 
six resourees, eaeh with five levels, five possible aetive goals with eaeh goal being in one 
of five states, and they ean be in any one of four eareer phases and any one of 10 types of 
plaees. From this, a soldier agent’s eontext may be any of 1.6xl0'^ eontexts. However, 
the important notion is that at any time, the soldier only has a single eontext. The agent’s 
eontext is eaptured by its extended eonneetors, whieh are managed loeally by the agent. 
In managing eonneetors (eontext) loeally, the agent self-regulates its interests and the 
faetors that infiuenee its aetions. Conneetors are the eommon thread that tie the state of 
the agent’s outer environment together with the inner environment, the agent’s goals, and 
through references, the appropriate tiekets and aetions to aehieve the goals. 

K. CONNECTOR-BASED SIMULATION MODEL 

The eonneetion aetion is at the heart of CMAS simulation. When a CMAS 
environment, ineluding all of its agents and objeets, is deseribed in terms of eonneetors, 
the eonneeting proeess provides a means of managing the eombinatorial explosion of 
possible states and eontexts, to faeilitate agent and objeet interaetion. In the eourse of the 
simulation, the agents aet on two levels, the first being the “reaetive” response based on 
eonneetor input. Stimuli reeeived through eonneetors trigger signaling easeades within 
the agent. The seeond is the aetion(s) the agent takes in pursuit of its goals. This aetion 
is the produet of the agent’s goal management proeess. The CMAS simulation model 
brings these two together. 
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While (not done) { 

Randomize agents 
For each agent { 

Sense environment 

SCAs and connectors sense environment 

Update existing bound connections (push and pull) 

Update Einner 

Connect 

Evaluate mfc function and initiate new connections 
Update new connections 

While (new connections exist) { 

Update new connections (push and pull) 

Update Einner 

Evaluate mfc function and initiate new connections 

} 

Act 

} 

} // end main loop 

Figure 28, CMAS Simulation Model 

L. SUMMARY 

This chapter presented a formal definition of the Conneetor-based Multi-Agent 
System (CMAS) and described a simulation model based on the CMAS arehitecture. The 
story engine presented in the next chapter is eonstructed from the CMAS architecture. 
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VI. INTERACTIVE STORY GENERATION SYSTEM 


A, INTRODUCTION 

The Interactive Story Generation System (ISGS) is comprised of three primary 
components that work in concert to structure and present interactive stories (Figure 29). 
These components are the story engine, scene rendering subsystem (SRS), and graphical 
interface. The ISGS follows the Model-View-Controller (MVC) architecture [Gamma et 
al., 1995]. MVC divides the system responsibilities into three parts: the model, which 
contains the program data; the view, which provides the visual presentation of the model, 
and the controller, which defines the system behavior. The story engine CMAS and 
associated data represent the controller and model, while the SRS and graphical interface 
provide the view. The graphical interface augments the controller by way of manual 
control in the form of user intervention and interaction with the stories. Following the 
MVC abstraction, replacing the SRS with another view can be performed without need to 
modify the story engine. 
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Figure 29, Interactive Story Generation System 
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This dissertation focuses on the story engine and story engine CMAS. First, this 
chapter relates story worlds and story lines to the CMAS architecture. The components 
of the story engine CMAS that uniquely set it apart from other CMASs are then 
presented. An inductive argument is presented as proof that the story engine generates 
story lines that are logically connected and goal-directed. Further, it is argued, that the 
story lines are sound with respect to the domain being modeled. Finally, the SRS and 
graphical interface are described as components of the integrated ISGS. The ISGS brings 
together the story engine CMAS with generative text-to-voice, photo-cell animation, and 
photo-realistic settings (locations) to present interactive stories. 

B, STORY ONTOLOGY 

This section describes the narrative constructs of story world, story and story line 
as they apply to this work. A one-to-one mapping is defined between these narrative 
structures and the corresponding CMAS constructs of the story engine. 

1. Story World, Story and Story Line 

A story world is comprised of characters and props, along with locations where 
events occur and characters interact with each other, the props, and their surroundings. 
The interactions are not random nor are the characters free to act as they wish without 
regard to the rest of the story world. There are constraints on character actions and 
interactions. Equation 28 defines a story world in terms of five components: characters, 
props, locations, actions and constraints. 

Story World = {Ch, P, L, A, Co} 

Ch - Characters 
P - Props 
L - Locations 
A - Actions 
Co - Constraints 

Equation 28, Story World Definition 

Consider the children’s story of The Three Little Pigs. In the story world of the 

three pigs, the world consists of three pigs, a wolf, hay, sticks, bricks, plus additional 

props (flute, fiddle, shovels, etc.) (Figure 30). The action takes place in multiple 

locations including a straw house, wood house and brick house. There are constraints on 

what the characters can do, though in this case they are very much controlled by the 
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author as opposed to constrained by physics and nature. Multiple story lines based on 
this story world are possible, but actually generating a story line requires that a set of 
initial conditions be set. In this case, the initial conditions might include the personality 
traits for the pigs and wolf, along with their skill levels and initial goal(s). Therefore, a 
traversal requires a story world plus a set of initial conditions. This tuple of {story world, 
initial conditions) defines a story. 

Continuing with the three little pigs, if the initial conditions define two happy-go- 
lucky pigs, one hard working pig and a hungry wolf, the result may be similar to the 
traditional story line of the three pigs. On the other hand, by modifying the initial 
conditions and defining three devious pigs with a good-natured wolf, it may be possible 
to get a completely different story line from the same story world, i.e.. The True Story of 
the 3 Little Pigs [Scieszka and Smith, 1989]. A story line is therefore the result of a 
traversal through a given story {story world plus initial conditions). 
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Three Pigs and a Wolf Story World 


Figure 30, Three Pigs and a Wolf Story World 
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The definition of the story world provides the “potential” for numerous and 
varying stories. When the story world is transferred to software that “potential” ean be 
realized in the form of dynamic story lines. 

2 . Story World as a CMAS 

The story world correlates directly with the definition of a CMAS (Equation 8). 
The CMAS components of agents, objects, and environment equate to the story world’s 
characters, props and locations respectively. The laws, operations and relations of the 
CMAS establish the constraints within which the characters interact. The goals provide 
the impetus for the characters while the tickets and actions provide the means for the 
characters to achieve their goals. When the CMAS is combined with a set of initial 
conditions, the result is a simulation. 

In this context, a simulation is a generating function that produces a story line. 
Let S be defined as a generating function that when applied to a CMAS produces a set of 
outputs, including a story line (Equation 29). 

Given 

CMAS M and initial conditions I, 

S is generating function that produces output as follows: 

S(M,I) = O; 
where 

O is a set of outputs and story line Le O. 

Equation 29. CMAS Generating Function (S) 

S takes initial conditions and a CMAS as input, and produces a set of outputs, 
including a story line, L. L is called a story line generated by S given initial conditions I. 
When S is left to run to completion without intervention, S is said to be non-inter active. 
If allowances are made for the user to intervene as the simulation progresses, as is the 
case with the story engine, then S is interactive. 

Repeated applications of S may generate identical or different sequences L each 
time S is applied with the same initial conditions. When repeated application of S 
generates identical story lines, S is called deterministic. Alternatively, when S generates 
a different story line each time it is applied, the simulation is said to be non-deterministic. 
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When S is interactive, it is likewise non-deterministic as a eonsequenee of the variability 
in user intervention. 

Unlike most stories produeed by systems with statieally defined plans, story 
engine CMAS models are purposely designed to produee novel or surprising story lines, 
i.e. non-deterministie story lines. There are two ways to do this: (1) by introdueing 
pseudo-randomness in the agent’s plans through Monte Carlo methods, or (2) by 
permitting the agents to devise their own plans as the story line unfolds. This dissertation 
distinguishes between Monte Carlo teehniques and dynamie plans, e.g. plans that are 
undefined initially, and unfold dynamieally as the simulation runs. 

The story engine is not eoncerned with finding a single plan or story line; it is 
intended to generate “possible” story lines. But rather than produee “possible” story lines 
through the applieation of Monte Carlo techniques, the story engine generates story lines 
by allowing agents to dynamically construct plans as they explore a story world, (as 
defined by the story engine CMAS data set) in pursuit of their goal(s). The resulting 
story lines are plans that lead to the achievement of a goal, and while achievement is 
significant, the path to achievement is also of interest. 

This agent-based approach to generating story lines does not exclude Monte Carlo 
techniques from being used to generate multiple replications in order to study the CMAS 
model of the domain. Examination of the frequency of possible story lines can give 
insight into the CMAS representation and also the likelihood of the answers occurring 
based on the agent’s representation of the domain being studied. 

The space of potential story lines is combinatorially large and therefore it is not 
possible to explore the entire space and visit every possible story line. The best that can 
be done is to conduct a focused exploration. In the previous chapter, it was shown how 
the focus of the agent is controlled via connectors. In essence, the connectors tune the 
agent’s attention to precisely those parts of the environment (agents and objects) most 
relevant given the agent’s current state; and allow it to communicate and coordinate with 
precisely those agents that can assist it in achieving its goals. This modeling approach 
focuses on the middle ground between finding a single answer and exploring all possible 
answers. 
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C. STORY ENGINE CMAS 

The story engine CMAS is a data-driven arehiteeture. The simulation domain is 
defined by the data, not the CMAS. In the next ehapter, a data set representative of U.S. 
Army eareer progression is deseribed. This data is used in America's Army: Soldiers to 
generate stories eentered on pursuing an army eareer. By ehanging data, the story engine 
CMAS ean be used to generate stories within a totally different domain. 

The story engine, as depieted in Figure 29, is a simulation kernel that implements 
the generating funetion S defined by Equation 29. The story engine is designed to 
operate on a speeifie instanee of a CMAS, called a story engine CMAS. The story engine 
is a non-deterministic, interactive engine that explores a story world, as defined by a story 
engine CMAS and associated data set, to generate story lines. The non-deterministic 
characteristic of the story engine results from user intervention and the possible use of 
pseudo-random numbers to make choices as the story line is generated. The interactive 
nature of the engine does not impact the repeatability of the story lines. If interaction by 
the user is considered part of the input, albeit occurring at varying points in the process, 
then interactively generated story lines are repeatable given identical initial conditions, 
random number seeds, and the same user interaction at the same times and places in the 
story line generation process. 

The story engine CMAS follows the general constructs of a CMAS as described 
in Chapter V. The unique components are two connector-based composite agent types; 
character agents and scene agents. The character and scene agents are capable of 
generating dynamic stories through a repetitive connection process. These two agent 
types are combined with tickets, actions, references, connectors, laws and relations to 
define a story world. Through the ISGS, the story world can be combined with imagery 
and sound to generate and visualize multiple story lines (Figure 31). 
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story Engine 


Scene Rendering Subsystem 



Figure 31, Interactive Story Generation System with Agents and Media 

The remainder of this section describes character agents, scene agents and the 
character-to-scene connection process. The following vignette is repeatedly referenced 
as the story engine CMAS components are described. 

A young man wants to buy a car, and on a Saturday afternoon he heads to 
the local used car dealership. The experienced used car dealer greets his 
customer and quickly sizes him up as a naive buyer. As they begin 
walking through the car lot, he assures the young man that he has the 
perfect car for him. The scene closes with the young man driving away 
in the “perfect car” having just signed a high interest loan, while paying 
well over the car’s value. 

1, Character Agents 

The story engine CMAS structures stories around a central character agent. This 
central character is designated the “main character” and is the protagonist of the story. 
The implication of this, from the CMAS perspective, is that connections with scene 
agents are always initiated by the main character. In the ISGS, the player’s character is 


103 






































the main character. In this way, the story is structured around the player’s interventions 
and their character’s goals. 

Character agents are composite agents, and as such, they are able to take full 
advantage of the capabilities provided by the connector-based composite agent 
architecture. The character agents described here closely reflect the agents developed for 
the initial version of the story engine CMAS constructed for the America’s Army: 
Soldiers project. They do not necessarily implement all of the connector-based 
composite agent features. For example, the character agents do not make use of 
Symbolic Constructor Agents; their external sensing occurs exclusively through extended 
connectors. This is not a limitation on the agents; it is simply how the initial version of 
the story engine CMAS was implemented. 

a. Goals and Actions 

The agent’s inner environment consists of a set of state variables with 
associated external connectors. The agents are equipped with a set of goals and a reactive 
agent to manage the goals. The agent has a set of tickets it uses to achieve its goals. As 
described in Sections IV.F and V.E.4, an agent’s goals resolve to specific actions through 
references and tickets (Figure 26). In the story engine, character agents have a limited set 
of actions that are primarily used to initiate the agent’s next connection. Just as actors 
don’t act until they are filling a role in a scene on stage (or in a movie), character agents 
can’t act until they are filling a role in a scene (connected with a scene agent). Because 
the characters are autonomous agents, they need control over which scenes they 
participate in. The character agent actions provide this control. An action’s precise 
function is to initiate a connection with a scene agent based on the connectors defined by 
the action. The actions that literally change the character’s state, further its goals and 
propel it through the story are resident in the scene agents. 

Once a connection is established, the character fills a role in the scene and is then 

free to act within the bounds of the scene based on its current state (as expressed through 

its connectors) and goals. For instance, if the character has a goal of buying a car, then it 

must connect with a scene that will put it in a position where it can purchase a car. The 

kind of car purchased, and how good a deal the character negotiates is influenced by how 

badly the character wants a car (goal weight) and how savvy the character is as a buyer 
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(state variable). The charaeter knows what it wants to do (buy a oar); oonnecting to a 
soene agent provides the how (oar lot soene). A detailed desoription of the oharaoter-to- 
soene oonneotion is provided in a following seotion. 

b. Most Favorable Connection 

When a oharaoter agent prepares to initiate a oonneotion, there are 
potentially many soenes it oan oonneot with, but it oan only oonneot with one soene at a 
time. That is, a oharaoter oan only be in one plaoe, doing one thing at a time. The 
oharacter agent initiates a oonneotion with a soene agent based on the evaluation of its 
most favorable oonneotion (mfc) funotion. As desoribed earlier, goals have a weight and 
a measurement method. The measurement method provides a numerio measure of how 
well the goal is being satisfied (higher being more satisfied), while the weight is a 
measure of the importanoe of the goal. These values are used by the mfc to identify the 
oonneotion that is “besf’ in terms of the agent’s goals. 

Given agent a, with goal set G, Equation 25 defines a set W'as the set of 
entities in the environment with whioh a is able to establish a oonneotion. With the story 
engine CMAS, oharaoter agents only oonneot with soene agents so If'oonsists entirely of 
soene agents. Equation 30 defines a oharaoter agent’s most favorable oonneotion 
funotion. The mfc is a measure of the goal’s weight (wt) times its level of satisfaotion, 
where satisfaotion level is 1.0 minus the goal’s measure (mm). The logio being that the 
more satisfied the goal, the less attention it needs at the ourrent time. This is sealed by 
the goal’s importanoe. 

Given agent a with goal set G and 
possible connections W'as defined by Equation 26, 

mfc = max (1.0 - mmi)(wti) 

gi^G 

s.t. conn(gi, JV) 

tnmi - gi’s measure; (0.0 <mmi<1.0) 
wti - gi’s weight; (0.0 <wti <1.0) 

(Bee W)conn(ai, e)—>conn(gi, W), 
where ai is the action bound to gi. 

Equation 30. Character Agent Most Favorable Connection Function 

Figure 32 shows a oharaoter agent’s goal struoture. Eaoh of the goals is 


bound to an aotion via a referenoe - some direotly, others through tiokets. The aotions are 
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bound to the references across a set of connectors (Equation 18, Figure 26). The bound 
connectors establish the set of connectors used by the mfc function to evaluate the 
possible connections. When the mfc is evaluated, the selected goal executes the bound 
action, which in turn initiates a connection with a scene agent based on the connectors 
identified by the action. In this example, a connection satisfying goal’s connection 
criteria (connectors a and P) has a value of (.8)(.9) = 0.72. Goab and goal’s connections 
have a value of 0.56 and 0.247 respectively. Assuming all three connections are possible, 
then goali’s connectors (a and P) would be used, along with the character agent’s other 
extended connectors, to connect with a scene agent. 


state measure (mm) weight (wt) action 



6 

Figure 32. Character Agent Goals with Bound Actions 


2, Scene Agents 

Scene agents are connector-based composite agents with components and 
behaviors appropriate for generating scenes of a story. When a character agent connects 
with a scene agent, a sequence of events is initiated in the agent that results in the 
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generation of a seene. Just as a story world provides the potential for many story lines, a 
seene agent provides the potential for many seenes. The eharaeter agents eonneeted with 
the seene agent internet within the bounds established by the seene agent’s tiekets, 
eonneetors (internal and external), and aetions in order to generate a seene. This seetion 
deseribes the eomponents and behaviors that are unique to a seene agent (Figure 33). 



Character Role Interaction ticket 


Figure 33, Scene Agent 

a. Scenes 

The story engine differentiates between a seene agent and a scene. When a 
eharaeter agent eonneets with a seene agent, an internal easeade is triggered within the 
seene agent that results in the generation of one element of the story line. This element of 

the story line is deseribed as a scene. The scene is eomposed of two parts: story engine 
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actions and scene rendering aetions. In the MVC arehiteeture of the ISGS, these two 
parts partition the seene into model/eontroller aetions (story engine aetions) and view 
aetions (scene rendering aetions). The story engine aetions are related direetly to the 
story engine CMAS and updating the agents’ states. These aetions are exeeuted during 
the eonneetion as the seene is being generated. The rendering aetions, on the other hand, 
are not exeeuted immediately; rather they are eaptured in a playlist to be exeeuted when 
the eonneetion is eomplete. The playlist is the interfaee between the story engine and the 
seene rendering subsystem. The seene, as displayed by the SRS, is an after-the-faet 
visualization of the aetions that oeeurred during the eharaeter agent to seene agent 
eonneetion. 

b. Inner Environment 

The seene agent has a set of state variables that deseribe the “roles” that 
must be filled for the seene. The roles are filled by eharaeter agents. In keeping with the 
story engine CMAS’s eharaeter-eentrie view of a story, one of these roles is always the 
main eharaeter role. The eharaeter initiating the eonneetion is automatieally oast in the 
main eharaeter role. The remaining list of roles is domain dependent. For example, in 
America's Army: Soldiers the roles inolude buddy, drill instruotor, oar salesman, etc. The 
roles oome into play as the scene unfolds. Just as with a movie or play, the role the aotor 
is oast into oonstrains what the aotor oan do. The oonstraints imposed by the story engine 
are muoh less restriotive than that of a linear movie or play. In traditional media, there is 
a fixed soript that aooompanies the role. The aotor’s freedom is limited to their range of 
talents in delivering their lines. With interaotive stories, the soript is not fixed and the 
eharaeter agent’s goals and state play a oritioal role in determining the outoome of the 
seene. 

c. Tickets 

The agent uses tiokets to assist in establishing a basio struoture for the 

seene. One of the tiokets is designated as the “seene tioket” and is used as a starting point 

for generating the seene. Figure 33 shows a seene agent with a three-frame seene tioket 

where eaoh frame oontains a referenoe (Equation 15). In this example, the tioket is 

logioally broken into a beginning, middle and end oorresponding to the setup of the 

seene, the main aotion and the oonolusion. The referenoes in eaoh of the frames conneot 
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to tickets or actions that move the action forward at run-time. This process is described 
later in this chapter in the Character Agent to Scene Agent Connections section. 

d. Connectors: External and Internal 

Scene agents use external connectors to sense the outer environment and 
maintain an up-to-date view of its inner environment. The scene agent’s external 
connector set is partitioned into two subsets. The first is the set of connectors maintained 
by the scene agent. The second is a set of connectors associated with each of the “roles” 
in the scene agent. The state and value of these “role” connectors are not established 
until the character agents connect and fill the roles. This late binding of connectors at 
run-time allows for a tremendously flexible and contextually sensitive connection 
process. A special type of action, called an interaction is specifically designed to take 
advantage of this delayed binding. The interactions, with their non-instantiated 
connectors, create a potential for action. However, that potential does not take shape 
until the character agents bind to a role. 

In addition, scene agents make extensive use of internal connectors to 
enable the signaling cascade that generates a scene. The internal connectors are a critical 
component of the agent’s internal control mechanism, used to manage its tickets and 
interactions. 

e. Interactions 

The car sales vignette described at the beginning of this section could have 
played out many different ways depending on any number of factors, including the car 
dealer’s personality and integrity, the buyer’s experience level, and how well the buyer 
liked the cars he was offered. Interactions are the components that enable this sort of 
variation; they use connectors (internal and external) as their primary control mechanism. 
When a scene is generated during a connection, it is generated on an interaction-by¬ 
interaction basis. 

Interactions are compound actions that bind together at run-time using 
internal connectors to generate complex and contextually appropriate sequences of 
events. They are comprised of two types of atomic actions: story engine CMAS actions 
and scene rendering actions. A strict partitioning is enforced to maintain the integrity of 

the story engine CMAS as a stand-alone controller system in the ISGS MVC architecture. 
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(1) Story Engine CMAS Actions. Table 1 lists the set of atomic 
story engine CMAS actions. They are broken into three categories: actions to update 
state variables (string and numeric), actions to control internal connectors, and an action 


to manage objects. 



Action Name 

Parameters 

Description 

State Variables 

SetStringVariable 

role 

state variable 
new value 

Set the state variable for the 
character agent filling the designated 
role to the string value specified by 
new value. 

SetNumeric Variable 

role 

state variable 
new value 

Set the state variable for the 
character agent filling the designated 
role to the numeric value specified 
by new value. 

IncrementV ariable 

role 

state variable 
delta 

Increment the state variable for the 
character agent filling the designated 
role by the specified delta value. 

Delta can be positive or negative. 

Connectors 

ExtendInternalConnector 

connector 

Change the state of the connector to 
extended. 

RetractIntemalConnector 

connector 

Change the state of the connector to 
retracted. 

Objects 

ObtainObject 

role 

object 

Add the object to the inner 
environment of the character agent 
filling the designated role. 


Table 1. Story Engine CMAS Actions 

This relatively simple set of actions is sufficient to update and 


maintain the character agent’s state. For example, as the car-buying scene is generated, 

the following actions would be among those executed: 

IncrementVariable (MainCharacter, bankBalance, -4000) 
ObtainObject (MainCharacter, car) 

The ObtainObject action will add the car object to the inner environment of the main 
character. In doing so, the buy-a-car goal will be satisfied and the state of the goal will 
change to achieved. There are additional actions associated with goals that may be 
executed when a goal changes state. Continuing with the car example, when the buy-a- 
car goal changes state to achieved, a recurring action is initiated that decrements the 


no 










bankBalance variable on a recurring basis to simulate a recurring car payment. The 
reactive agent controlling the character agent’s goal structure manages this action. 

(2) Scene Rendering Actions. Table 2 lists the set of atomic scene 
rendering actions with a brief description. These actions are processed by the animation 
engine, text-to-voice, and location generator to visually render a scene. A detailed 
explanation of each action is found in [Elzenga, 2001]. 


Action Name 

Description 

SetLocation 

Generate a background. 

PlaySound 

Play an ambient sound. 

Stop Sound 

Stop playing an ambient sound. 

StopAllSounds 

Stop playing all ambient sounds. 

PlayAnimation 

Construct an animation. 

PlaySentence 

Generate a line of dialog. 

PlayMovie 

Play a movie. 

SetExitCondition 

Set conditions for ending animations. 

AddScreenTransition 

Set type of screen transition. 


Table 2, Scene Rendering Actions 

(3) Interaction Sequences. Interactions provide a mechanism for 
conditionally executing story engine and scene rendering actions based on stimuli from 
Einner. They providc a wrapper around groups of atomic story engine CMAS and scene 
rendering actions so they execute in coordination. For example, when the car dealer 
greets the young man, a single interaction animates the car dealer, animates the young 
man and plays contextually appropriate dialog for the greeting. The interaction is played 
out against a background that is appropriate for the time of day and with landscape 
appropriate to the general region where the character is located. Additionally, during the 
interaction a connector extended on the car dealer agent senses the experience level 
connector on the buyer that causes the dealer to update a state variable indicating he is 
dealing with a naive buyer. This state change in the car dealer impacts the remaining 
generation of the scene. 

Interactions have three components: a control function (/^), a set of 
connectors (Q and a set of actions (A) (Equation 31). 
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Interaction i = <fc, C, A> 

fc - control function 
C - set of connectors (C ^Ci u Ce) 

A - set of actions (A cAse uAsr) 

Ase is the set of story engine CMAS actions 
Asr is the set of scene rendering actions 

Equation 31. Interaction Definition 

The control function is responsible for extending and retracting the 
interaction’s connectors, initiating a connection to continue the interaction sequence, and 
controlling the execution of its atomic actions (Equation 32). The interaction’s 
connectors, when extended in receptor mode, are used to establish the conditions under 
which it can execute. The interaction’s connectors are a subset of the scene agent’s 
internal and external connectors, including connectors associated by role. After the 
interaction executes, it extends internal connectors in stimulus mode to express what the 
interaction just did. 

Given, 

Interaction intXi = <fc(i), Cu Ai> 

Interaction intxj = <fc(j), Cj, Aj> 
where, 

C'i ^Ci is intXi ’s set of extended connectors (receptor mode) 

C'j ^ Cj is intXj ’s set of extended connectors (stimulus mode) 

then intXj connects with intXi (conn(intXj, intx)) iff: 

(Vcj ^ Cj)Bci e C'i (conn(cj, c)) 

(conn(Cj, Ci) is defined by Equation 6) 

Equation 32. Interaction-to-Interaction Connection - conn(intXj, intxi) 

Figure 34 depicts a simple greeting, and response to a greeting, as 
an illustration. In this example, the main character greets the supporting character. The 
greet interaction is initiated from the first frame of a ticket. After the greeting executes, it 
extends an internal “greet” connector. The two “greet respond” interactions both require 
a “greet” connector, but they are conditional on the mood of the supporting character. In 
this case, the angry response will be executed. The response interaction will in turn, 
extend a “greet respond” connector, but with no interactions able to respond to the “greet 
respond,” the interaction chain ends and the next frame of the ticket is executed. 


112 




This was a simple example to demonstrate the most basie funetion 
of interaetions. In praetiee, the happy and angry responses would be eombined into a 
single interaetion that would exeeute eonditionally based on the state of the supporting 
eharaeter. In the next ehapter, a more substantial illustration is presented dealing with 
Army training. 
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Figure 34, Greet / Response Interaction Sequence 
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As previously stated, when an interaetion exeeutes, it extends an 
internal eonneetor. Other interaetions ean respond to this eonneetor and attempt to 
exeeute. If more than one interaetion tries to exeeute, the eurrent approaeh is to 
randomly seleet the interaetion. This does not oeeur very often sinee the interaetion’s 
eonneetor set ensures only eontextually appropriate interaetions attempt to exeeute. The 
random seleetion tends to provide random variation among appropriate ehoiees. 
However, the random method could be modified using a weighting function similar to the 
most favorable connection function (Equation 26 and Equation 30). 

This process of extending connectors, and responding, creates a 
dynamic and contextual chain reaction of interactions not unlike the signaling cascade 
phenomenon found in cells (Eigure 13 and Eigure 25). The next section demonstrates 
how interactions are combined with tickets to harness the contextual chain reaction and 
generate scenes. 

3, Character Agent to Scene Agent Connections 

Story lines are generated by the story engine through a repeating process of 
character agents connecting with scene agents. As previously described, there is a central 
character agent called the main character that initiates connections with scene agents 
based on its goals as determined by its most favorable connection function. 

Eigure 35 depicts the connection process and resulting generation of a playlist, 
along with a post-connection view of the interaction trace. In Eigure 35(a) the main 
character agent is evaluates its possible connections and ranks them with respect to its 
goals. In Eigure 35(b), the agent initiates a connection with a scene agent. As a result of 
the connection, the “main character” role is filled. This triggers a cascade, beginning with 
the scene agent extending a connector to initiate a connection with an agent to fill the 
supporting character role. Once the roles are populated, the scene ticket begins executing 
(Eigure 35(c)). The scene ticket initiates the process by which interactions chain together 
to form contextually appropriate sequences of events. As a result, the state of both the 
main character and supporting character are updated (story engine CMAS actions), and a 
playlist is generated (scene rendering actions) (Eigure 35(d)). The tree structure in Eigure 
35(e) is an after-the-fact portrayal of the interactions (and encompassed actions) executed 


114 



by the agents as the connection progressed. This structure was generated based on the 
characters’ goals and state, as opposed to having been laid out in advance and selected for 
a list of predefined plans. 





main 

character 

supporting 

character 


I ) playlist 
<—^ interaction 


Figure 35, Character Agent to Scene Agent Connection 

This next figure extends the single connection/single scene to a multi- 
connection/multi-scene story line generated as the main character repeatedly initiated 
coimections in pursuit of its goals (Figure 36). 
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Figure 36, Story Line Generated from a Story Engine CMAS Story World 

The story line in Figure 36 is tantamount to a dynamically generated plan. The 


scenes were generated one connection at a time as the main character agent pursued its 
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goals. The resultant plan evolved through the run-time binding of tiekets and aetions 
using the agent’s aetive eonneetors. A depth first traversal of the tieket/scene tree 
explains how the agent achieved its goal, while the connectors provide insight as to why 
the agent generated the plan in the way that it did. 

D, LOGICALLY CONNECTED, GOAL-DIRECTED AND SOUND STORY 

LINES 

This section presents an inductive argument as proof that the story engine 
generates story lines that are logically connected and goal-directed. Further, it is shown 
that the story lines are sound with respect to the domain of interest. 

1. Logically Connected and Goal-Directed Story Lines 

By induction over the length of the story line, it is shown that story engine 
generates logically connected, goal-directed story lines. 

Let s(n) be a story line of length n generated by the story engine generating 
function S(M, I) (Equation 29). s(l), a story line of length one, is generated as a result of 
character agent a connecting with scene agent s. 

The connection is established as a function of agent a’s state and agent 5 ’s state, 
as expressed through connectors (Equation 22). 

Given agent a with awareness set W (Equation 24), the set of candidate 
connections W' is a function of both a’s state, and the state of the scene agent’s in W 
(Equation 25). 

Result 1. The set of scene agents that agent a can potentially connect with (IE) is 
established as a function of agent a’s state. 

Selection of scene agent sj e W' as the connection to initiate is the result of 
evaluating character agent a’s most favorable connection function {mfc). The mfc is a 
function of agent a’s goals and extended connectors (Equation 30). 

Result 2. Given a set of candidate connections, agent a initiates a connection with 
a scene agent based on its goals, and state as expressed through extended connectors. 

During the connection, sequences of interactions execute to generate a scene. 
Interaction sequences are self-generated based on the state of character agent a, which 
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initiated the eonneetion, the state of seene agent s, as well as the state of the eharaeter 
agents filling the supporting eharaeter roles (Equation 32). 

Result 3. The established eonneetion progresses in a logieally eonneeted manner, 
based on the state of all agents involved (as expressed through their eonneetors). 

From results (1), (2) and (3) above, s(l) is a story line that is logieally eonneeted 
and goal-direeted. 

s(l) is the produet of a eonneetion with a seene agent, where the eonneetion is 
established aeeording to a eonneetor-based, goal-direeted seleetion funetion. The 
eonneetion was initiated from a eandidate eonneetion set of seene agents, where 
membership in the set was established as a funetion of agent a’s state and the state of the 
seene agents in a’s awareness set. Further, the eonneetion progressed in a logieally 
eonneeted manner, based on the state of all agents involved. 

Assuming that s(n-l) is a logieally eonneeted and goal-direeted story line, s(n) is 
likewise logieally eonneeted and goal-direeted. 

Sinee s(n) is generated in sequenee, the story line ean be deeomposed into the first 
n-1 seenes eoneatenated with a story line of length one: 

s(n) = s(n-l) + s(l). 

If s(n-l) is logieally eonneeted and goal-direeted and the same is true for s(l), 
then s(n) is a goal-direeted and logieally eonneeted story line of length n. 

2. Sound Story Lines 

The story engine is a simulation engine. As sueh, it is used to evaluate software 
implementations of eomputational models. The eomputational model, when eombined 
with a data set, is an abstraetion of a real world system. The story engine CMAS is one 
sueh eomputational model that formulates the real world system in terms of eharaeter 
agents, seene agents, tiekets, eonneetors, interaetions, and story engine aetions. 

Equation 29 established the story engine as a generating funetion (5) that takes as 
input a story engine CMAS {M), along with a set of initial eonditions (7), and generates 
output that ineludes a story line L. This ean be expressed as F e S(M, I). 
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The story engine is said to be sound with respeet to a domain of interest if S(M, I) 
generating a story line L, implies that T is a story line from the domain of interest. Let D 
be a domain of interest, then S(M, I) is sound with respeet to D if 

LeS(M, I)^LeD. 

The story engine CMAS is a purely data-driven arehiteeture, where the eharaeter 
agents, seene agents, tiekets, ete., are instantiated solely from data, not by the CMAS 
arehiteeture. If D is a real-world domain of interest, let Abe a story engine CMAS data 
set used to model D, and Mx be the story engine CMAS instantiated from A. A deseribes 
the domain D in terms of eharaeter agents, seene agents, tiekets, eonneetors, interaetions, 
and story engine aetions. The CMAS data representation of O is a model, and as sueh, it 
is an abstraetion of the real world domain. Sinee A is an abstraetion of D, let Dx be the 
aetual domain modeled by A. 

In the previous seetion, it was established that the story engine generating 
funetion generates story lines that are eonneeted logieally based on agent eonneetions and 
eonneetor state. Agent eonneetions are established, and eonneetor state is maintained, by 
the story engine CMAS eonstruets, strietly as defined by the data. Therefore, 

L eS(Mx,I) -^LeDx, 
and S(Mx, I) is sound with respeet to Dx. 

If A aeeurately represents the temporal, struetural, and proeedural relationships 
and eonstraints of D, then Dx ^D. 

If (Dx^D) A (Le Dx), then (Le D). 

. •. (Dx^D) -^[L e S(Mx, I) e D]. 

So, if the CMAS data set A aeeurately deseribes the domain of interest D, then the story 
engine S(Mx, I) is sound with respeet to D. 

E, RUN-TIME ANALYSIS AND SCALABILITY OF STORY LINE 

GENERATION 

Story lines are generated on a eonneotion-by-oonneetion basis as initiated by the 
main eharaeter agent. In this seetion, it is shown that for a given story engine CMAS M, 
there are a faetorial number of possible story lines. In addition, a run-time analysis is 
presented that shows a story line of length k is generated in 0{kca) where c is the number 
of eonneetor types and a is the number of agents in the story engine CMAS. 
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1, Run-time Analysis 

Let M be a story engine CMAS with a agents (a; seene agents and a 2 eharaeter 
agents) and a eonneetor set with c eonneetor types. At any given step of the story line, 
the main eharaeter agent (a^c) could have at most c eonneetors extended and aj seene 
agents in its awareness set. If Umc has c eonneetors extended, then to determine if it ean 
eonneet to an agent a, e ai, it must eompare eaeh of its c eonneetor types with the 
matehing type (if extended) on a,. This eomparison is a eonstant time 0{1) operation. 
So, in the worst ease, Umc is able to evaluate its eonneetion with a, in time 0{c). If there 
are ai agents, eaeh with c eonneetors, then the main eharaeter requires at most cai 
eomparisons to determine whieh seene agent to eonneet with. 

Onee a eonneetion is established between the main eharaeter agent and the seene 
agent, the seene agent must then establish eonneetions with the supporting eharaeter 
agents. There are at most a 2 -l eharaeter agents to fill the supporting eharaeter roles (one 
of the eharaeters is the main eharaeter). The seene agent must make at most c(a 2 -l) 
eomparisons to fill its supporting eharaeter roles. Therefore, at eaeh step of the story line 
generation, the number of eomparisons to initiate all required eonneetions and begin the 
seene generation proeess is c(a/) + c{a 2 -l), whieh is equal to c{a-l), or 0{ca). 

Seenes generate by exeeuting a subset of the seene agent’s set of internal 
interaetions. Sinee eaeh interaetion ean exeeute at most onee, this proeess oeeurs in 
eonstant time. Therefore, a single element of the story line is generated in 0{ca), and a 
story line of length k, is generated in 0(kca), where c is the number of eonneetor types 
and a is the number of agents in the story engine CMAS. 

2. Story World Dimension 

Let N{M) represent the number of story lines possible from the story engine 
CMAS M and let N{M, k) be the number of possible story lines of length k. It is shown 
that A(M) is 0{aj\), where ai is the number of seene agents inM. 

Given the set of ai seene agents, if it is assumed that eaeh seene agent ean be 
eonneeted with at most onee, and that all orderings and eombinations are possible, then 
the number of possible story lines of length k {k < aj) is equal to the number of k- 
permutations from a set of aj distinet seene agents, denoted P{ai, k). This is, however, an 
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upper bound because the actual number of possible story lines is a function of the 
temporal constraints on their ordering and combination as implied by tickets and 
connectors. Therefore, 


N{M,k)<P{a„k). 
Summing across story lines from length 1 to a; yields; 

k=a^ k=a^ 

Y^N{M,k)<Y,P{a„k) 


_ ^ a\'- 

-kV- 

k=ai^ -I 

= ai!y- 

^(ai -k)\ 

N{M) = 0{ay\) 

Therefore, given a story engine CMAS M, there are a factorial number of possible 
story lines, and for any given story line of length k, the story engine generates the story 
line in time 0{kca), where c is the number of connector types and a is the number of 
agents in the story engine CMAS. 

F. SCENE RENDERING SUBSYSTEM 


The Interactive Story Generation System (ISGS) combines the story engine with a 
scene rendering subsystem (SRS) and a user interface to present and control interactive 
stories (Figure 29). The SRS and graphical interface were developed independent of the 
story engine by programming teams supporting the America’s Army: Soldiers project. In 
its current state, the ISGS allows for user intervention between scenes but not during the 
scenes, resulting in interactive stories, but not interactive scenes. While the story engine 
is capable of supporting intra-scene user intervention, the SRS, at present, is only able to 
render scenes in their entirety. 

The SRS provides an after-the-fact view of the actions that occurred during the 
agent-to-agent connection. The story engine and SRS interface via a playlist generated 
by the scene agent. The playlist is a compilation of SRS specific commands used to 
render the scene. The SRS is comprised of three integrated components; an animation 
engine that controls the characters’ movements on screen, a location generator that 
creates contextually appropriate photo-realistic backgrounds and a text-to-voice system 

that combines prerecorded dialog pieces in accordance with a narrative grammar. 
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1, Animation Engine 

The animation engine operates on an event-based, time relative structure for 
media playback. In this scheme, animations are not scheduled for play at strict times or 
for a predefined number of frames. The length of the animation is dependent on outside 
cueing, such as sound fdes beginning to play, sounds fdes completing, or keystrokes and 
mouse clicks. In the opening of the car lot scene, the car dealer greets the potential 
buyer. From the SRS perspective, this entails animating the car dealer speaking, 
animating the buyer listening and playing a greeting dialog. The greeting should be 
contextually appropriate, so the exact series of sound files to be played is not set until the 
greeting interaction is executed. As a result, the length of time the greeting dialog plays 
is variable. In order to accommodate varying length dialog, the character animations 
must play in synch with the spoken dialog. Sustained animation, synchronized to outside 
cueing, has been used successfully in a number of applications including the Microsoft 
Office 97 office assistant [Kessler and Kilgore, 1997]. 

The SRS animation engine constructs animations from individual frames of 
digital imagery. Actors are filmed against a blue screen going through a range of motions 
and gestures. The raw media is processed and a portion of it is used to construct a 
database of animation frames. Much like a cartoon animator brings a character to life 
through a sequence of still images, the animation engine uses the same technique to 
animate a character agent using individual frames of digital imagery. For a given 
animation action, the engine selects a set of prescribed frames from the database and 
plays them in accordance with action-specific ordering and branching rules to generate 
the animation. Each character has a fixed database of frames. But, once the frame set is 
established and cataloged, the frames can be reused in many different animations. A 
relatively modest set of frames can be used to produce hundreds of on-screen actions. A 
detailed description of the animation engine can be found in [Elzenga, 2001]. 

2. Location Generator 

Eocations are the stage on which the story action takes place. Erom a single data 
set, the story engine is capable of generating hundreds of scenes being played out in 
many different settings (locations). Rendering the scenes requires the ability to situate 
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stories and characters in contextually appropriate and believable locations. The location 
generator is responsible for constructing backgrounds that are appropriate for the given 
scene and also fit the character. It manages a database of reusable media pieces to 
construct the locations at run-time (Figure 37). 

A typical location consists of six to ten media pieces that include a sky, mid¬ 
ground and foreground, context-specific props, and character animations. If the character 
is at a beach in southern California, the background should reflect a smooth sandy beach. 
On the other hand, if the same scene occurs while the character is in New England, the 
background should include a rocky shoreline. The location generator populates location 
templates at run-time with media that is appropriate for the character’s current state. The 
location generator makes use of the character agent’s connectors when populating the 
template. Details of the instantiation process can be found in [Elzenga, 2001]. 
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Figure 37, Location Templates 

3, Text-to-Voice 

The process for constructing character dialog is similar to constructing locations. 
With locations, a template is used to guide the selection of images used to create the 
background. In the case of the text-to-voice system, a generative grammar is used to 
define sentence structures. These structures included variables appropriate to the scripts 
and the domain. Voice actors record full sentences that are then split into atomic pieces 
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and placed in a database. The text-to-voiee system’s sentenee generator parses sentenee 
definitions and eonstruets the dialog from the database of atomie speeeh files. 

Figure 38 shows an example of a greeting sentenee for the ear dealer seene. 
Based on the time of day, gender and age of the eustomer, and the dealer’s analysis of the 
eustomer’s experienee level, an appropriate greeting is eonstrueted. The greeting can 
range from “Good morning ma’am, ean I help you?” to “Hi there young man. Weleome 
to Billy Bob’s. This is your lueky day. I just reeeived a oar that you are going to love.” 
[Elzenga, 2001] provides a full desoription of the text-to-voiee system and its integration 
with the animation engine. 

<greeting> :: <salutation;parameters><title:parameters><weloome;parameters> 

<salutation>[parameters: time of day] 

Good morning [between 0800 and 1200] 

Good afternoon [between 1200 and 1800] 

Good evening [between 1800 and 2200] 

Hello [random] 

Hi there [random] 

<title>[parameters: gender, age (as determined by ear dealer)] 
sir [male, 30 or over] 
ma 'am [female, 30 or over] 
young man [male, under 30] 
young lady [female, under 30] 

<weleome>[parameters: buyer’s experienee level as determined by ear dealer] 

Can I help you? [experieneed] 

What can I do for you? [random] 

Welcome to Billy Bob’s. This is your lucky day. Ijust received a car that 
you are going to love. [ naive] 

Figure 38, Text-to-Voice Generative Grammar 
G. GRAPHICAL INTERFACE 

The graphieal interfaee provides the interaetive link between the story engine and 
the user. As a simulation engine, the story engine is eapable of generating story lines 
without intervention. However, by allowing the user to internet with the agents, the story 
engine is able to present a story line that is highly personalized. In the ISGS MVC 
arehiteeture, the interfaee augments the automated eontrol provided by the story engine 
CMAS with a “manual” eontrol meehanism. The level of interaetion is applieation 
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dependent. The user ean take on a highly involved first person role as a character in the 
story, or step back and influence the story via high-level adjustments to the story world’s 
relations and laws. In America’s Army: Soldiers, presented in the next chapter, the user 
guides their character by adjusting the character’s goals and values, but they do not have 
direct control over the character’s actions. The character remains an autonomous agent. 
H. SUMMARY 

The ISGS structures and presents interactive stories through an instance of a 
CMAS called a story engine CMAS. The story engine CM AS is a data-driven 
architecture that when combined with a domain specific data set, generates dynamic, 
goal-directed story lines. The SRS processes playlists generated by the story engine’s 
scene agents, to present a view of the story as it unfolds. Through a graphical interface, a 
user is able to interact with the agents and objects of the CMAS story world. 

The story engine’s story lines are assembled on a connection-by-connection basis 
as a by-product of character agents pursuing their goals. The generation of a story line is 
equivalent to the construction of a historical tree that records the derivation of the agent’s 
plan to achieve its goal. A depth-first traversal of the tree provides an explanation of how 
the agent reached its objective. 
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VII. CMAS DATA SET IMPLEMENTATION - AMERICA’S 

ARMY: SOLDIERS 


A, INTRODUCTION 

The story engine was fielded as the underlying simulation engine in an 
interaetive, story-based role-playing game for the U.S. Army. This ehapter introduees the 
America’s Army projeet, and deseribes the story engine CMAS data set used to 
interaetively generate story lines focused on army career progression. The tickets, 
connectors, scenes, and characters, including their goals and personalities, are presented. 
The America’s Army: Soldiers project is offered as a proof-of-concept of this dissertation 
research. 

B. AMERICA’S ARMY: SOLDIERS 

America’s Army is a suite of two applications developed for the U.S. Army 
intended to provide young people with accurate, easy-to-assimilate information about the 
Army so they can develop a better understanding of army life and available opportunities. 
The two applications, America’s Army: Operations (AA: Ops) and America’s Army: 
Soldiers (AA: Soldiers) offer a realistic portrayal of Army training, missions and values. 

AA: Ops is a first-person perspective gaming environment that demonstrates life 
in the infantry. The experience begins with required individual training, such as basic 
rifle marksmanship and Basic Combat Training (BCT). Once qualified, the player is able 
to take part in multi-player missions. AA: Ops focuses primarily on combat training and 
tactical missions. While these are critical, they represent only a small part of army life. 

AA: Soldiers is a sophisticated role-playing game that provides a look at Army 
personnel and career opportunities by allowing the player to guide a character through a 
career in the Army. It presents the Army through stories that are sensible yet surprising, 
interactive, and non-repeating. In order to visually portray Army bases, offices, barracks 
and facilities as accurately and faithfully as possible, digital photographic and video 
imagery was chosen as the display medium for the Scene Rendering Subsystem. 

A story engine CMAS data set was constructed to accurately capture the Army 
domain in terms of career progression, general training and specialty schools, as well as 
on and off-duty life. 
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Interaction in the game commences at character definition time, and continues 
throughout the game by adjusting the character’s “core values” and goals. The player 
cannot exert direct control over the character, i.e., cannot make a specific decision for the 
character. The character is an autonomous agent; as such, the player must guide their 
character through the story world by intermittently adjusting its goals and values. 

C. INTERACTIVE STORY GENERATION SYSTEM DATA 

The Interactive Story Generation System data is partitioned into two sets, data 
used to define the AA: Soldiers story world, and data necessary to visually render the 
scenes (Figure 39). The story world is defined by a story engine CMAS data set 
comprised of three primary parts: character agent definitions, scene agent definitions, and 
object definitions. Connectors from the character agent’s inner environment and goal set, 
as well as a small collection used by the scene agents to describe the setting of the action, 
are combined to make up the AA: Soldiers connector set. This set is described later in the 
chapter. The Scene Rendering Subsystem is instantiated from data that is specific to each 
of the modules: speech, animations and locations. 
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Locations 



Figure 39, AA: Soldiers Interactive Story Generation System Data Set 
D, CHARACTER DEFINITIONS 

Characters inAA: Soldiers are defined aeeording to their personality, aptitude and 
goals. Personality is delineated in terms of the Army’s seven “eore values” [U.S. Army, 
1999], while aptitude is measured aeeording to six “resouree” levels. A eharaeter is also 
bestowed with a set of goals, whieh are managed by the player. In addition, eharaeter 
agents are instantiated with a set of tiekets and notions refleotive of Army oareer 
progression. In terms of story worlds and story lines, the main eharaeter’s personality, 
aptitude and goals establish the initial oonditions for the story world. These attributes 
also oombine to drive the oharaoter’s behavior and, ultimately, the story line. While 
tiekets provide general story direotion, i.e.. Army oareer progression, the oharaoter’s 
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goals influence its path through the story world. The character’s aptitude, as determined 
by its resources, determines how quickly the character progresses toward achieving its 
goals. They provide fuel for the character. Finally, personality establishes how steadfast 
the character is in working towards its goals. Low core values can result in the character 
losing focus and being led astray. 

This application is designed to introduce young people to all aspects of the Army, 
including providing them with a view of what the Army expects of its recruits and 
qualified soldiers. Misaligned values, resources and goals can attract the character to 
scenes that exploit out-of-balance conditions. 

1. Actors, Characters and Roles 

The SRS uses digital imagery and recorded audio to animate the characters and 
construct their dialog. As such, a set of “actors” were filmed and photographed against a 
blue-screen, and recorded in a sound studio to generate the required images and audio. In 
conjunction with the actors, the story engine maintains a list of recognized “roles.” These 
roles include main character, supporting character, drill instructor, car salesman, and so 
on. Associated with a role is the requirement to take certain physical actions, i.e., look 
left, point right, kneel down, etc. Roles such as main character and supporting character 
are very general and require a wide range of actions. Other roles are more specific, such 
as the salesman, and require a smaller range of actions. Since not all actors have the 
digital frames to support every possible action, an actor definition file is used to delineate 
the roles the actor is capable of filling. In the case of the main character “role,” the 
player selects an “actor” and sets values, resources and goals to define their “character.” 
Figure 40 depicts the screen used to define the player’s “character,” which fills the main 
character “role.” The remaining actors are mapped to characters by the story engine. 
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Figure 40, AA: Soldiers Character Definition Screen 
2, Core Values and Resources 

A character’s personality is defined in terms of the Army’s seven eore values: 
loyalty, duty, respect, selfless serviee, honor, integrity, and personal eourage (Figure 41). 
There is a state variable eorresponding to eaeh of these that ranges from zero to 100. 
Assoeiated with eaeh variable, is a eonnector whose control function graduates the 
variable into live levels ranging from “low” to “high.” As the variable ehanges value, the 
connector changes state aeeordingly. When a new game is initialized, the player defines 
a eharacter, whieh includes distributing a limited budget of “value points” among the 
seven eore values. As the game progresses, additional points are aeeumulated, or lost 
based on the eharaeter’s aetions. The player intervenes as desired to distribute newly 
accumulated points, or readjust the eurrent eaehe of points. 
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Figure 41, Character Personality Defined by Army Core Values 

Six resources are used to drive the character: energy, skill, strength, knowledge, 
financial, and popularity (Figure 42). Like the core values, the connectors associated 
with each of these are graduated from “low” to “high.” Flowever, unlike core values, the 
player cannot directly adjust the resources. Resources increase and decay based on the 
character’s actions and achievements. 



Figure 42, Character Aptitude Expressed in Terms of Resources 
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3, Goals 

AA: Soldiers' goal set is widely varying, ineluding goals targeted at sueeeeding in 
the Army, and others representative of the target age group of 18 to 24 year olds. There 
are 60 goals in the applieation that deal with personal and professional growth (Table 3). 
The goal set was arrived at through joint eonsultation between the AA: Soldiers projeet 
manager and Army’s projeet sponsor. Personal goals range from party, date, and buy a 
ear, to establish eredit, eollege degree and time management. The professional goals 
eover the major areas of eareer, soldiering and maturing. Career goals deal with solving 
work problems, ereating solutions, and multi-tasking. Maturing goals eover aspeets sueh 
as dealing with authority, following direetions and paying attention to detail. Soldiering 
goals foeus the eharaeter on developing professional skills required of a soldier, sueh as 


marksmanship, map reading and meeting physieal training requirements. 


Have Fun 

Party 

Be Cool 

Minimize Work 

Pay Attention to Detail 

Deal with Authority 

Teamwork 

Work Hard 

Study Skill 

Advanced Study Skills 

Oral Communications 

Good Vocabulary 

Writing Skills 

Advanced Writing Skills 

Physical Exercise 

Physical Training (PT) Program 

Amateur Sports 

Marksmanship 

Advanced Marksmanship 

Personal Luxuries 

Stereo 

TV 

Computer Games 

Home Entertainment Center 

Car 

Sports Utility Vehicle 

Euxury Car 

Date 

Relationship 

Marriage 

Baby 

College AA Degree 

College Bachelors Degree 

College Graduate Degree 

Professional Certificate 

Qualify PT Run Time 

Qualify PT Push-ups 

Qualify PT Sit-ups 

PT Badge 

Advanced PT 

Extreme PT 

Map Reading 

Visualize Map 

Reading Skills 

Reading for Speed 

Advanced Reading Skills 

Time Management 

Multi-tasking 

Meet Deadlines 

Follow Directions 

Solve Work Problems 

Create Solutions 

Foreign Language 

Establish Credit 

Pay Bills 

Savings Account 

Manage Money 

Live Off Post 

Read Technical Manuals 

Trouble Shoot Problem 


Table 3, Character’s List of Potential Goals 


Unlike most agent-based simulations, where the agent maintains its own goal 
priorities, in this applieation the player is responsible for seleeting and prioritizing the 
their eharaeter’s goals. The player seleets and prioritizes up to five goals (Figure 43). 
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Figure 43, Goal Emphasis and Progress 

Goals have preconditions for selection, requirements that must be met to achieve 
the goal, and a set of results or effects of achieving the goal (Figure 44). Preconditions 
are defined by a set of prerequisite goals that must be achieved before the goal is 
available to be selected. This relationship creates a “goal hierarchy,” a portion of which 
is shown in Figure 45. The result of achieving a goal includes unlocking new goal 
opportunities, adding to, or subtracting from resource levels, and strengthening or 
weakening values. The goal’s progress (measure) and emphasis (weight) are used to 
drive the player’s experience as they progress through the game. 
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Figure 44, Goal Description Including Prerequisites, Requirements and Results 



Figure 45, Goal Hierarchy Formed from Goal Prerequisites 
4, Tickets: Army Training Progression 

Army training progression, particularly in the early stages of a soldier’s career, is 
highly structured (Figure 46). This structure is reflected in the tickets defined for the 
character agents. Table 4 provides a list of the primary tickets used to structure the 
character’s progression through the initial stages of Army training. These relatively 
simple tickets are sufficient to structure approximately the first nine months to a year of a 
recruit’s life in the Army. At the current time, this period of training is the primary focus 
of AA: Soldiers. The list described in Table 4 is only a partial list, in that for each of the 
Army occupational specialties, there are tickets that capture the unique elements of the 
training regimen. Basic Combat Training (BCT) and the initial stages of One Station 
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Unit Training (OSUT) are similar for all career specialties. However, Advanced 
Individual Training (AIT) and the later stages of OSUT are specific to the operational 
specialty selected. 


r 


Combat Arms 

^ _ 




Non-Combat Arms 


Figure 46, U,S. Army Career Training Progression 


Ticket 

Description 

MBPS 

Military Entrance Processing Station: An initial processing stage 
required for all military recruits. 

BCT 

Basic Combat Training: Initial training for recruits entering non¬ 
combat arms specialties. 

AIT 

Advanced Individual Training: Follow-on training to BCT that is 
related directly to the recruit’s chosen specialty. 

OSUT 

One Station Unit Training: Basic and specialty training combined 
into a single program for combat arms specialties. 


Table 4, Army Training Progression 

Once a character enters their first duty assignment, the career progression 


possibilities begin to expand. Just as with any profession, future opportunities are 
normally a function of past performance. In the Army, there are a number of educational 
and professional opportunities that are based on performance. Some of these include 
Airborne training. Ranger training, and Special Operations. These schools, and paths to 
advanced designations, are structured as tickets, with connectors used to enforce the 
prerequisites in terms of core values (integrity, personal courage,...), resources 
(knowledge, strength,...) and experience, such as AIT complete and Airborne qualified. 

E, CONNECTOR SET 

This section describes the AA: Soldiers connector set. It is a compilation of 
connectors used to capture character state. Army training progression, and backdrops for 
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the scenes (Table 5 and Table 6). As indicated earlier, an AA: Soldiers character is 
described according to their core values, resources and goals (Table 5). Army career 
progression is represented by occupational specialty, post (Army base), career phase, and 
duty phase (Table 6). Career and duty phase combine to model the stages of an Army 
career. Within the career phases, particularly the training segments, there is a well- 
defined structure that is represented by the duty phase connector. Figure 46 depicts the 
career phases from initial enlistment up through the soldier’s first operational duty 
assignment. Sixteen military occupational specialties (MOS) and fourteen posts are 
represented in the game. Finally, there are connectors that assist in establishing the stage 
setting (location) where the action takes place and scene plays out (Table 6). 



Connector Type 

Connector Values 


Loyalty 

Low, MediumLow, Medium, MediumHigh, High 

(/) 

0 

Duty 

II 


Respect 

II 

> 

Selfless Service 

II 

CD 

L_ 

o 

Honor 

II 

6 

Integrity 

II 



Low, MediumLow, Medium, MediumHigh, High 


Energy 

Low, MediumLow, Medium, MediumHigh, High 

(/) 

CD 

Strength 

II 

L_ 

Knowledge 

II 

o 

(/) 

Skill 

II 

<D 

01 

Financial 

II 


Popularity 

Low, MediumLow, Medium, MediumHigh, High 


Have Fun 

Selectable, Selected, Achieved 


Party 

II 

(/) 

CO 

o 

■ 

II 

o 

Marksmanship 

II 


Teamwork 

II 


Manage Money 

Selectable, Selected, Achieved 


Table 5. Connectors Describing Character Personality, Aptitude and Goals 
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1 Career Progression Connectors | 

I Loction Connectors | 

MOS 

Post 

Duty Phase 

Career Phase 

Location Group 

Place 

Infantry 

Combat Engineering 
Field Artillery 

Air Defense Artillery 
Special Forces 
Armor 

Signal Operations 
Electronic Maintenance 
Chemical 
Ammunition 
Administration 
Petroleum and Water 
Medical 

Supply and Services 
Military Police 
Military Intelligence 

Ft. Benning 

Ft. Bliss 

Ft. Bragg 

Ft. Eustice 

Ft. Gordon 

Ft. Hood 

Ft. Huachuca 

Ft. Jackson 

Ft. Knox 

Ft. Lee 

Ft. Leonard Wood 
Ft. Sam Houston 
Ft. Sill 

Ft. Wainwright 

Arrival 

Indoctrination 
Phase 1 
Phase 2 
Phase 3 
Qualified 
Graduation 

Recruiter 

MEPS 

BCT 

AIT 

OSUT 

Operational Duty 
Discharged Honorable 
Discharged OTH 
Retired 

Formation 
Work Detail 
Training Station 
Field Training 
Off Duty 
Classroom 

PT Field 

Rifle Range 

Gym 

Office 
Barracks 
Apartment 
Game Room 
Weight Room 
Garage 

Aircraft hangar 
Parking Lot 
Obstacle Course 
Commissary 

PX 

Chapel 

Bank 

Car Lot 

Lake 

Jail 

Pawn Shop 
Barracks 
Restaurant 
Airport 


Table 6, Connectors Describing Army Career Progression and Locations 
F. SCENE DEFINITIONS 

Scenes are defined in terms of interaetions, tiekets, internal eonneetors and roles 
(Figure 33). This seetion demonstrates, by example, the seene generation proeess, and 
how the outeome of the seene is a funetion of the main eharaeter, as well as the eharaeters 
filling the supporting roles. The following portrays the main eharaeter attempting to 
qualify in the run portion of the physieal training (PT) test during phase two of basie 
training. 

1. Exemplar Scene: Physical Training 

The outeome of a seene ineludes not only the rendering and result (quit, fail or 
eomplete the run), but also the ehanges that oeeur to the eharaeters partieipating in the 
seene. Figure 47 depiets the in-proeess eonneetion between the main eharaeter and PT 
seene agent, with the supporting eharaeter eonneetion already established. Table 7 
eatalogs the interaetions from the PT Seene agent, ineluding the eonneetors (internal and 
external) and aetions. The seene progresses aeeording to the three-frame seene tieket in 
the seene agent. The first and last frames deal with setting and elearing the stage (“Set 
Stage” and “Clear Stage” respeetively). These two aetions are not deseribed sinee they 
deal almost exelusively with the Seene Rendering Subsystem. 
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The main action of the scene is initiated in the second frame of the scene ticket 
with the “Start Run” interaction (Table 7). This scene is meant to challenge a character 
with marginal personal courage and strength; characters with medium to high personal 
courage and medium to high strength complete the run without problem (Table 7; 
Finish(l)). A main character with low personal courage or low strength could possibly 
complete the run, provided they get some encouragement from the supporting character 
(Table 7: Encourage, Finish(2)). On the other hand, a supporting character with low 
selfless service can discourage the main character causing them to quit (Table 7; Quit). 

As the scene generation progresses, the values and resources of the characters are 
updated, leaving them in a new state once the agents disconnect. In this example, the 
supporting character discourages the main character, causing the main character to quit. 
The result is lower strength and personal courage on the part of the main character and 
lower selfless service and loyalty on the part of the supporting character. 


After Connection - Before Execution 



After Execution 



Scene Ticket 


Set 

Start 

Clear 

Stage 

Run 

Stage 




Figure 47, Basic Physical Training: Run Qualification Scene 
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Interaction 

Connectors 

Actions 

(^^^art Ru^ 

IC: CueStartRun 

IC: Run Started 

(^inish(^ 

IC: Run Started 

MC: Strength (Med - Hi) and 
Personai Courage (Med - Hi) 

IC: Finish 

MC: Personal Courage +2 
Strength +2 

(^^DoubtAbilit^ 

IC: Run Started 

MC: Strength (Low) or 

Personal Courage (Low) 

IC: Doubt 

(^^^courag^ 

IC: Doubt 

SC: Self Svc (Med - Hi) 

IC: Encourage 

SC: Loyalty +1 

Selfless Service +1 

MC: Personal Courage +2 

(^^^Discourag^ 

IC: Doubt 

SC: Selfless Service (Low) 

IC: Discourage 

SC: Loyalty-3 

Selfless Service -2 

MC: Personal Courage -2 


IC: Encourage 

MC: Personal Courage (Low) 
Strength (Low) 

IC: Quit 

MC: Personal Courage -1 
Strength -3 

Loyalty +1 

Quit^^ 

IC: Discourage 

IC: Quit 

MC: Strength -3 

Loyalty -1 

(^inish(2)^ 

IC: Encourage 

MC: Strength (Med - Hi) or 

Personal Courage (Med - Hi) 

IC: Finish 

MC: Strength +2 

Loyalty +3 


IC: Internal Connector, MC: Main Character, SC: Supporting Character 


Table 7, Interactions for Basic Physical Training: Run Qualification Scene 

2, Exemplar Story 

An exemplar story is provided in Appendix A. The description begins with the 
player’s selection of an actor and definition of a character, and continues through the 
initial scenes of a basic training story. 

G. OBJECTS 

In this application, objects are associated closely with goal achievement. There is 
a relatively small set of objects, relating to possessions the character can accumulate or 
long term commitments on the part of the character (Table 8). These possessions and 
commitments carry with them recurring costs, either financial or in terms of time, or 
both. For example, buying a car results in recurring car payments. Failure to meet the 
commitments can lead to difficulties. On the other hand, successfully managing the 
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commitments leads to rewards in terms of strengthening the eharaeter’s values and 
resources, and exposing additional goal opportunities. 

Car 

Sports Utility Vehiele 
Luxury car 
TV 
Stereo 
Computer 
Video Games 

Home Entertainment Center 
Steady Relationship 
Spouse 

_ Baby _ 

Table 8, AA: Soldiers Story World Objects 

H, SUMMARY 

AA: Soldiers provided a unique opportunity to combine researeh with applieation. 
It was a collaborative effort involving a team of nine artists, sound teehnicians and 
programmers working on various media produetion and software development efforts 
associated with the Interactive Story Generation System. The efforts included 
development of the animation engine, location generator, text-to-voice system and 
interface, with parallel and eomplimentary efforts to generate the underlying media. 

The AA: Soldiers projeet serves as proof-of-coneept that the CMAS arehitecture, 
and story engine CMAS, deseribed in this dissertation are eapable of generating 
interactive stories. 
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VIII. CONCLUSIONS AND RECOMMENDATIONS 


This chapter summarizes the research presented in this dissertation. It begins by 
highlighting the major eontributions, and is followed by a description of possible avenues 
for application. While this researeh made contributions to the field of interactive stories 
and multi-agent system simulation, it also raised a number of important questions. 
Aeeordingly, the chapter eoncludes with a discussion of recommendations for future 
work. 

A, CONCLUSIONS 

The story engine provides a fundamentally new approach for generating 
interaetive stories. The underlying eoneept moves away from centralized eontrol, and 
generates stories as a by-product of agent interactions through the distributed arehitecture 
of a multi-agent system. The story engine is a robust simulation engine that is not tied to 
any single domain or display medium. The domain independent nature of the story 
engine is a eharacteristic inherited from the general CMAS architeeture. The bottom-up 
design employed by the story engine allows it to scale to large story worlds, and generate 
story lines from a faetorially huge story spaee. With the protagonist-eentrie exploration of 
the story world, where the protagonist responds to the user’s interventions, there is a 
strong sense of perceived consistency in the story lines as the user guides their character’s 
journey. 

The story engine CMAS definition provides a formal and descriptive notation for 
translating an abstract story world into software. Using sueh an arehiteeture, researchers 
ean explore, manipulate and stress story worlds through the use of agents. 

The formal deseription of a Connector-based Multi-Agent System (CMAS) 
arehiteeture, along with aeeompanying description of an agent eommunication, 
coordination and control process based on connectors and connecting, provides an 
avenue for researchers to investigate the capabilities of connector-based simulation. 

The proof-of-concept implementation of the story engine in AA: Soldiers 
demonstrates the feasibility of the CMAS arehitecture, and specifieally, the instanee of 
the architeeture as defined by the story engine CMAS. The speeifie components and 
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operations of the story engine CMAS eapture the requisite elements to describe a story 
world and, more importantly, generate story lines. 

B, APPLICATIONS 

Returning to the initial motivation for this research, founded in the belief that 
there is value to be gained from “linking entertainment and defense,” the applications for 
interactive story are resident in both domains. The story engine provides an advanced 
simulation methodology for training and education, as well as for story-based interactive 
entertainment. While the story engine blurs the lines between simulation and 
entertainment, this research leaned more toward defense application at the possible 
expense of immersive narrative entertainment. 

1. Educational Gaming 

The America’s Army: Soldiers project is a game with a message. The Army 
recognizes that in order to attract young people to the military, it must first educate them 
as to what the Army has to offer. In this sense, the story engine is very much an 
information packaging and presentation tool. At the same time, it is important to present 
the message in a form that is appealing to the target audience (i.e., 18 to 24 year olds). 
On this front, the entertainment value of the engine takes center stage. By combining 
accurate information with a game-like interface, and presenting it in a personalized story 
format, it is possible to achieve an engaging, educational, and entertaining experience. 

2. Scenario-Based Training 

Scenario-based training involves the use of scenarios to help people better 
understand the decisions they have to make on a day-to-day basis. It is particularly 
effective for situations where there is no single right or wrong answer. When presented 
correctly, scenario-based training goes beyond what can be found in “the manuals” and 
challenges the trainee with events anchored in lessons learned and on-the-job experience. 
Allowing a trainee to explore paths that may lead to undesirable outcomes is oftentimes 
more valuable than simply providing the answer. Recognizing the correct path is a 
matter of both knowledge and experience. In its current form, the story engine, and entire 
ISGS, might be utilized as a general-purpose interactive scenario-based training system. 


144 



C. FUTURE WORK 

1. Narrative Structure 

The story engine constructs stories that follow logical cause and effect 
relationships, and are goal-directed. It adheres to the constraints of the domain and 
generates well-structured interactive story lines detailing the character’s passage through 
the story world. The original expectations of this research were to develop a highly 
scalable simulation engine to meet the above objectives, while at the same time following 
a pronounced narrative structure. This research concluded with the first goal met, and 
makes progress towards the second. Chapter II described a number of narrative 
structures used in modern-day screenplays. These structures provide a design model for 
defining software narrative templates. A character agent’s most favorable connection 
function might be modified to rate candidate connections, not only according to goal 
weight, but also according to their fitness with respect to a narrative template. By 
influencing the character agent to connect with scene agents that are not only within it’s 
set of candidate connections, but also aligned with a narrative template, stories with a 
more pronounced narrative structure will be possible. 

2. Potential Outcome Modeling 

The CMAS architecture provides an avenue for capturing complex domains in 
software. It makes it possible to model a domain of interest and explore the space using 
goal-directed agents searching for “interesting outcomes,” where an “outcome” is 
described as an achieved goal, plus the path taken by the agent to achieve the goal; and 
“interesting” is defined by a domain-specific metric. By creating agents with malevolent 
goals, it may be possible to explore a domain, and thereby identify and exploit unforeseen 
weaknesses. In addition, it is possible to recreate the path of choices to reach the goal by 
following the agent’s sequence of connections and connectors. 

3. Improved Cognitive Architecture 

The character agents employed in America’s Army: Soldiers make use of a very 
simple cognitive architecture comprised of personality and aptitude state variables (core 
values and resources respectively). This representation is adequate for modeling 
character behavior with respect to a game, but it is not sufficient for modeling realistic 
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human performance and behavior. Realist performance is a function of imperfect 
perception, reduced cognitive processing and behavior that is not always optimal. These 
imperfections are often times due to a lack of information, misperception of stimuli, or 
limited cognitive resources. The result is reduced, yet realistic, human performance. 
[Wellbrink, 2002] is exploring the use of a complex adaptive system as the basis for a 
cognitive architecture that models reduced human performance. The story engine will 
benefit from the incorporation of an improved cognitive architecture for the character 
agents, particularly when the agents are used to explore complex domains for potential 
outcomes. 

4. Relations 

[Roddy and Dickson, 2000] provides a detailed discussion of relations as they 
apply to multi-agent systems. Included in this work is the description of an architecture 
for managing inter-relationships among agents called RELATE. Reformulating 
RELATE as a connector-based architecture, and incorporating it into connector-based 
composite agents will provide a notable enhancement for modeling relation-centric 
domains, such as those involving military command structures. 

5. Generalized Connecting 

In the current implementation of the CMAS architecture and story engine, 
connections only occur across type-matched connectors. This constraint might be relaxed 
to allow connectors to connect based on generalization relationships (i.e., via a 
superclass/subclass or superinterface/subinterface hierarchy). This generalization 
supports object-oriented design patterns and will permit connectors to connect with 
“kinds” of connectors vice matching types. When moving away from modeling highly 
structured domains such as military career progression, it may also be beneficial to 
explore the use of fuzzy sets as a basis for establishing connections [Zadeh et al, 1996]. 

6. Agent Learning 

Tickets provide a means of capturing procedural knowledge. By allowing agents 
to augment their ticket set during a connection, procedural knowledge (i.e., experience) 
can be passed from one agent to the next. Eor example, when an agent connects with a 
training scene, additional tickets that capture the learning objectives of the course could 
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be passed to the agent. This sort of tieket passing might be used to simulate deeentralized 
or interactive rote learning as described by [Weiss, 1999]. 

D, SUMMARY 

This chapter presented the major contributions of this work: a general-purpose 
multi-agent system simulation architecture based on connectors, and a scalable 
simulation architecture for generating interactive stories. Significant areas for future 
work still remain, both in connector-based multi-agent simulation, and in interactive 
stories. From British novelist Mary Augusta Ward [Columbia, 1996]: 

The first law of story-telling... Every man is bound to leave a story better 

than he found it. 

With anticipation, this research leaves both fields of study better than when they 
were found. 
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APPENDIX A. EXAMPLE STORY 


This appendix is provided to depiet the initial scenes of a story session. It begins 
with the player defining a character and selecting a military occupational specialty 
(MOS). Next it presents, in chronological order, the character’s arrival at basic training, 
initial wake-up the first morning, along with classroom and field training sessions. 







1 Main Play 

Continuout Play la ON 

Contact tha - 

Help ’ 


hheI 


Select the actor you wish to play your soldier. 



•J 
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Choose a name for your soldier. 


u'looie 

Park 

Perez 


Rodriguez 

Schmidt 

Sutherland 


Powers 

Adjust your soldier's Values, Goals and Resources. 

VALUES GOALS 


RESOURCES 


Create your soldier for this game. 


The player selects an actor and 
defines their character. 
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' CANCEL PROCEED 


Adjust your character's values. 






the seven core values. 
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The character’s initial goals are 
selected and prioritized. 


r Main _Pl^r_ Continuein PUiy i» ON _ Contict tfte ^ i Help 

AR MY 

A bar means a resource is declining. GREEN indicates advancing, and YELLOW means stable. 


Energy 

Strength 

Knowledge 

Skill 

Finanf^ial 

Popularity 


2$ 90 re 


0 25 90 75 100 


0 25 SO 75 100 


0 25 SO 75 100 


Powers 


The character’s resources are 
adjusted. This is possible at 
definition time only. Once the 
story begins, it is no longer 
possible to manually adjust the 
resources. They increase and 
decay based on the character’s 
actions and achievements. 


view your character's accumulated resources. 



The player and character meet the 
recruiter. 
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The player selects the military 
occupational specialty of Field 
Artillery for their character. 




Continuou* Pliy i» ON Contact tli 


The character is sent to basic 
combat training (BCT). 


Welcome 

to 

Basic Training 


r- 



The character arrives at BCT and is 
welcomed by the drill instructor. 
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The recruits awake to the sound of 
a screaming drill instructor. 



In this classroom training scene, 
the player’s character has low 
energy resources and is falling 
asleep during the drill instructor’s 
lecture concerning the general 
orders of a sentry. 



The drill instructor wakes the 
player’s character, and tells him to 
stand in the back of the classroom 
in order to stay awake. 
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The player must adjusts their 
character’s goals and values in 
order to encourage the character to 
focus on their low resources. The 
player tries to gain access to the 
character by clicking on the door 
of a quickly spinning ball. As the 
player gains their character’s trust, 
the ball begins to slow down. 



The character’s goal of physical 
exercise will help to increase his 
strength and energy. 



The recruits are complimented by 
their drill instructor for 
successfully completing an 
assigned task. 
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The character’s initial attempt to 
qualify at the rifle range does not 
go well. He needs to pay closer 
attention to detail. 


The character gets some 
encouragement and advice from 
his buddy before the next attempt 
to qualify. 



The character successfully 
qualifies on his second attempt. 
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The recruits proudly display their 
targets to the drill instructor. 
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APPENDIX B. ACRONYMS AND ABBREVIATIONS 


AA: Ops 

AA: Soldiers 

ABL 

ABT 

AI 

AIT 

A-Life 

BCT 

CA 

CARTE 

CMAS 

DAI 

DoD 

E 

inner 

Eouter 

ICT 

ISGS 

MAS 

MEPS 

MEC 

MOS 

MRE 

MUD 

MVC 

NRC 

OSUT 

PAM 

PT 

RA 

SAM 

SCA 

SRS 

UME 

use 

VTP 


America’s Army: Operations 
America’s Army: Soldiers 
A Behavioral Eanguage 
Active Behavior Tree 
Artificial Intelligence 
Advanced Individual Training 
Artificial Eife 
Basic Combat Training 
Composite Agent 

Center for Advanced Research in Technology for Education at USC 

Connector-based Multi-Agent System 

Distributed Artificial Intelligence 

Department of Defense 

Inner Environment 

Outer Environment 

Institute for Creative Technologies at USC 

Interactive Story Generation System 

Multi-Agent System 

Military Entrance Processing Station 

Most Eavorable Connection 

Military Occupational Specialty 

Mission Rehearsal Exercise 

Multi-User Dungeon 

Model-View-Controller 

National Research Council 

One Station Unit Training 

Plan Applier Mechanism 

Physical Training 

Reactive Agent 

Script Applier Mechanism 

Symbolic Constructor Agent 

Scene Rendering Subsystem 

Unified Modeling Eanguage 

University of Southern California 

Virtual Theater Project at Stanford University 
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